Changelog, Monday 12th August 2024

More forms, more fetchers, functional tests, document classification, and scraper analysis.

Our most productive week yet, at least according to one, easy-to-game metric:

We are taking more time than I'd like on the form, but that's my fault: I wanted a full end-to-end codegen form builder. It seemed silly to me to specify the validation logic in three different places. We are also using codegen extensively, including for the API server interfaces. So we grafted some libraries together and got it working 98% of the time.

But, as with so many of these things, it wasn't the 98% that mattered–it was the 2%, the long tail of idiosyncratic behaviours, that sunk it. So we went back to being a little more explicit and a little less DRY.

We ran our local plan scrapers for the full country, which taught us a lot, and it took a day or so to sift through everything. Our intern Zubin then started working through the rejects.

The other big piece of work completed this week was the classification worker, an important part of our early document processing pipeline.

More website work in the few minutes between; now waiting to get the snag list back from the design team. And I scoped out the Appraisal design refresh work, which will begin next week.