Grant Proposal - Apache Tuweni for a complete execution client

Moloch Rises
#1

Apache Tuweni

Website

Twitter / Social

Type: Project

Application

Apache Tuweni is an Apache Foundation software project which defines multiple libraries for Ethereum and Secure Scuttlebutt. Tuweni has an Ethereum client in the works.

Last year, Tuweni benefited of a grant from MolochDAO to develop a JSON-RPC proxy server. This year, Tuweni is closer to becoming compatible with mainnet, with its own EVM, its own blockchain repository and its ability to sync.

EVM

The Apache Tuweni EVM is implemented currently for the Berlin hard fork, and it will require some retrofitting to cover every hard fork of Ethereum. Using the reference tests maintained by the EF, it sounds possible to achieve.

Syncing

Apache Tuweni can sync headers and block bodies, but doesn’t have a good way to validate them by replaying transactions. That’s the next logical step of the process: work towards syncing mainnet with the client.

UI

Apache Tuweni is a bit different from other clients in the sense that it can sync multiple chains at once and is provided with a web user interface, allowing operators to start/stop, replay syncs. The UI right now is non-existent besides exposing the client config. This would be the next step to support running syncs and operating Tuweni.

Performance

Apache Tuweni is not tested for the volume of transactions, data and peers coming with mainnet. It will require some performance work to make sure it’s able to succeed in keeping with head and optimize its storage.

Team

Antoine Toulme - http://antoine.toulme.name

‍Working on Ethereum protocol since 2017, on Pantheon and Hyperledger Besu.

Teku, Geth, Hyperledger Fabric, Corda contributor

Moloch grantee for Eth2 testing

EEA chair of the Testing and Certification Working Group

Requested Funds (USD)

50,000

Budget/Scope Details

Implement 2 deliverables by the end of three months:

  • A complete JSON-RPC server and client proxy

  • A transaction pool and signer

Q & A

‍

Will you be working on this full-time? Or how are you planning to balance this work with other projects?

‍I have multiple active engagements at this time:

  • I hold a full time position as engineering manager of the blockchain and DLT team at Splunk.

  • I have engaged in multiple, targeted engagements to help blockchain communities:

I am currently receiving a grant from the Ethereum Foundation to bring to production a network crawler.

I am actively supporting Hyperledger Besu for Ethereum classic, specifically implementing Keccak mining and the Magneto hard fork.

‍

Any plans to get other contributors involved? (a client will never get off the ground with a single maintainer)

‍Yes, and that’s precisely why it’s an Apache project: https://twitter.com/_tmio/status/1400621727621677058

Here is the list of current contributors:

https://people.apache.org/phonebook.html?podling=tuweni

Of course, there is a steep imbalance right now:

We voted a new committer a month or so ago, and will continue onboarding people.

https://cwiki.apache.org/confluence/display/TUWENI/Board+reports

https://cwiki.apache.org/confluence/display/TUWENI/May+2021

‍

Is Tuweni influenced at all by recent innovations like those in Erigon?

Yes, very much so. That said, we took a modular approach from the start.

The Ethereum client for example may run multiple chains at once, and we can choose to run components separately.

This is a bit different from what Erigon is doing, but we’ll be able to interface when they have stable GRPC interfaces.

We also try to bring the best of the old world of software.

We use Infinispan for storage and caching, which allows active-active clustering and replication across machines.

https://infinispan.org/docs/9.4.x/user_guide/user_guide.htmlhttps://infinispan.org/docs/9.4.x/user_guide/user_guide.html#which_cache_mode_should_i_use*

‍

Is Tuweni following the merge developments at all? that’s coming up fast

Yes, we’re following the developments.

Yes, it’s coming fast.

It will still take quite some time to crystallize.

Tuweni might be well suited to embed PoS directly as a consensus engine.

‍

I look forward to your feedback. I am obviously very interested in understanding your own challenges and interests.

Please feel free to educate me on where you would like to have the most impact.

Relevance

  • More developers

  • More client diversity

  • Better composability - peel off a component you need to just use JSON-RPC or a network crawler

  • A great Apache licensed client without corporate control (the Apache Software Foundation forbids corporate interests)

How did you find out about us?

Prior grantee

Beneficiary Address

atoulme.eth

#2

Hey Antonie, I recall seeing your previous grant proposal and I just wondered about the progress of that, if there is any relation to this grant, would this be a connecting piece of infrastructure, were you able to deliver the milestones from the previous grant ?

Happy to see you back and pushing things forward :pray:t4:

#3

Yep, I delivered the JSON-RPC proxy library and server. See my interview with the DAO there:

1 Like
#4
  • A complete JSON-RPC server and client proxy

Wan’t that already competed?

  • A transaction pool and signer

A complete client sounds like a lot more work. How far along the path to a useable client does this project go? How much more work would be needed to deliver something that many people could use, and is there a realistic path to building and maintaining that?

#5

The proxy is done, yep.
A complete client is more work, but I haven’t stopped working on this. I have now sync working and I have implemented all opcodes for most hard forks, but I have bugs by the bucket.
I need to implement a block validator and processor, a transaction processor and validator, for all versions of each hard forks.
I need to implement the JSON-RPC interface and map all the methods to the blockchain objects.

It’s a bit unwieldly. Here is what I’m working on right now: GitHub - atoulme/certification
This is a generator for tests for each opcode of the EVM.