Start a 'test' directory and add a bunch of async tests#519
Conversation
|
Nice! And thanks for writing these up! Do you have ideas on how to improve the longevity of these tests over time? For example ideally we'd run these tests in CI but that's not easily possible today as wasip3-prototyping still needs bug fixes. Even if that weren't the case though it'll always be true that spec things will likely land here first and, in theory, have tests before it's implemented anywhere. For the time being it seems reasonable to me to land them here (and somehow set up infrastructure to have them work their way over to Wasmtime), and I have no issue with not running them for now. In the long-term though I do think that we'd ideally get to the point of being able to run the tests. The "most viable" path at this current juncture would possibly be something like depending on Anyway, hence leading to the question about thoughts of running these tests in CI here somehow! |
|
That's a great question, and I was thinking along the same lines. Ideally, we would simply have a reference interpreter in the same repo and CI would run the test suite on it. #192 actually has a draft reference interpreter, but it's a bit stale now. But, as you said, I have been considering whether I should fill out the existing Canonical ABI Python into a full reference interpreter. It would take some time, though, and getting to a 1.0 feature set has so far been the priority. But once a few more key missing pieces are filled in, I could maybe switch gears and do that. |
alexcrichton
left a comment
There was a problem hiding this comment.
Oh I meant to approve this as well, and yeah makes sense to me. In the meantime I like the idea of prepping various bits of infrastructure to get these tests plumbed around.
|
Alright, looks like everything succeeds on the current dev release of wasip3-prototyping with the exception of the one "TODO" in |
This PR adds a bunch of tests to a new
test/asyncdirectory, along with a few placeholder tests intest/valuesandtest/resources. The async tests call all the built-ins and exercise a number of the major semantic cases, but of course many more tests are necessary to cover all the semantically interesting cases.Currently, the tests found a number of small differences between the spec and the implementation in the
wasip3-prototypingrepo, which have been filed and are in the process of being merged. Once the merging is complete, the tests should all successfully run on the implementation.If anyone wants to get into writing more wast tests, please jump in! There's a good amount of example material to work off of to get started.