Discussed with:
- infinisil- ekleog- samueldr- abathur- gchristensen Store alongside arch + attrpath + nixpkgs revision - unknown (for archs not built by hydra, or nixpkgs revisions that were in-between two hydra evals)- manual-cannot-be-made-to-build (eg. if upstream doesn't support it)- build-broken- builds- passes-tests (passthru.tests)- manual-tests-passed (for manually testing)- manual-tests-failed (for when manual testing shows something that passes-tests actually doesn't work) if unknown, fetch last known build status if passes-tests, look if history looks like (passes-tests)* manual-tests-passed,and if so say how long it's been (if not, say for how long it's beenpasses-tests since the last lower-level build) if build-broken, look if history looks like (build-broken)*manual-cannot-be-made-to-build, and if so say how long it's been (if not, sayfor how long it's been build-broken since the last higher-level build) database should be auto-filled by hydra + committers could also manually fillstatuses, esp.:- build-broken -> manual-cannot-be-made-to-build- passes-tests -> manual-tests-passed, manual-tests-failed- unknown -> any other level (so exotic archs know what works) improving on an arch would basically be taking builds that are build-broken butdon't have a manual-cannot-be-made-to-build history, and trying to either fixthem or to mark them as manual-cannot-be-made-to-build Further ideas:- hydra could compute a probability of success based on the history of the build, to change the likelihood it has of rebuilding a package, thus economizing some compute- user-facing nix could use said probability (instead of the above “last known status + metadata” idea) to evaluate working likelihood Other things that could be stored in a metadata database that'd work for workinglevels:- build time (compensated by machine speed)- number of reverse-deps (maybe? if it's cheap enough it could inform people trying to work on most important packages for new archs)- number of cache downloads for this derivation, maybe? (or maybe it'd be better in the popcon db below?) Things that might be better not having in this working-level-centered metadatadatabase:- popularity contest (popcon): it's tied to the “package” and not to the derivation itself. Maybe set a meta.id or similar that'd be the package identifier, and maybe default to ${pname}-${version}- (in popcon db) list of known users (if they agree to?) to poke for testing new big package changes -> "I notice you're using the new version of jq. I don't have any reports on this yet. Can you try it out and run blahblahblah if it appears to be working fine?" (if it notices the jq attrpath being used and some option is set to report that to a central server) // requires more thought for abusive traffic though…- (in popcon db) maintainer list (because it's not actually related to the code and could change?)- (in popcon db) flag package for “needs more automated tests”, auto-suggest that users send out commands that provide tests for the packages (and maybe even a wrapper that auto-straces and asks the user if it's ok to send this data?) // maybe this doesn't actually need a flag but just to look for manual-tests-failed?