Snippet #4247 (by Ekleog, rst)

Expires in: 0 minutes Compare Snippets View Raw
  1. Discussed with:
  2. - infinisil
  3. - ekleog
  4. - samueldr
  5. - abathur
  6. - gchristensen
  7.  
  8. Store alongside arch + attrpath + nixpkgs revision
  9.  
  10. - unknown (for archs not built by hydra, or nixpkgs revisions that were
  11.  in-between two hydra evals)
  12. - manual-cannot-be-made-to-build (eg. if upstream doesn't support it)
  13. - build-broken
  14. - builds
  15. - passes-tests (passthru.tests)
  16. - manual-tests-passed (for manually testing)
  17. - manual-tests-failed (for when manual testing shows something that passes-tests
  18.  actually doesn't work)
  19.  
  20. if unknown, fetch last known build status
  21.  
  22. if passes-tests, look if history looks like (passes-tests)* manual-tests-passed,
  23. and if so say how long it's been (if not, say for how long it's been
  24. passes-tests since the last lower-level build)
  25.  
  26. if build-broken, look if history looks like (build-broken)*
  27. manual-cannot-be-made-to-build, and if so say how long it's been (if not, say
  28. for how long it's been build-broken since the last higher-level build)
  29.  
  30. database should be auto-filled by hydra + committers could also manually fill
  31. statuses, esp.:
  32. - build-broken -> manual-cannot-be-made-to-build
  33. - passes-tests -> manual-tests-passed, manual-tests-failed
  34. - unknown -> any other level (so exotic archs know what works)
  35.  
  36. improving on an arch would basically be taking builds that are build-broken but
  37. don't have a manual-cannot-be-made-to-build history, and trying to either fix
  38. them or to mark them as manual-cannot-be-made-to-build
  39.  
  40. Further ideas:
  41. - hydra could compute a probability of success based on the history of the
  42.  build, to change the likelihood it has of rebuilding a package, thus
  43.  economizing some compute
  44. - user-facing nix could use said probability (instead of the above “last known
  45.  status + metadata” idea) to evaluate working likelihood
  46.  
  47. Other things that could be stored in a metadata database that'd work for working
  48. levels:
  49. - build time (compensated by machine speed)
  50. - number of reverse-deps (maybe? if it's cheap enough it could inform people
  51.  trying to work on most important packages for new archs)
  52. - number of cache downloads for this derivation, maybe? (or maybe it'd be better
  53.  in the popcon db below?)
  54.  
  55. Things that might be better not having in this working-level-centered metadata
  56. database:
  57. - popularity contest (popcon): it's tied to the “package” and not to the
  58.  derivation itself. Maybe set a meta.id or similar that'd be the package
  59.  identifier, and maybe default to ${pname}-${version}
  60. - (in popcon db) list of known users (if they agree to?) to poke for testing new
  61.  big package changes -> "I notice you're using the new version of jq. I don't
  62.  have any reports on this yet. Can you try it out and run blahblahblah if it
  63.  appears to be working fine?" (if it notices the jq attrpath being used and
  64.  some option is set to report that to a central server) // requires more
  65.  thought for abusive traffic though…
  66. - (in popcon db) maintainer list (because it's not actually related to the code
  67.  and could change?)
  68. - (in popcon db) flag package for “needs more automated tests”, auto-suggest
  69.  that users send out commands that provide tests for the packages (and maybe
  70.  even a wrapper that auto-straces and asks the user if it's ok to send this
  71.  data?) // maybe this doesn't actually need a flag but just to look for
  72.  manual-tests-failed?

Reply to this snippet →

Honeypot, don't fill.
⌘+⏎ or Ctrl+⏎