Commit graph

117 commits

Author SHA1 Message Date
Ben Kohler e86116180e
catalyst-auto: extra echo to debug CLST_AUTO_NOCLEAN
Signed-off-by: Ben Kohler <bkohler@gentoo.org>
2024-04-02 06:53:43 -05:00
Ben Kohler bd05aec8be
catalyst-auto: allow skipping cleaning with CLST_AUTO_NOCLEAN var
Signed-off-by: Ben Kohler <bkohler@gentoo.org>
2024-03-19 11:14:11 -05:00
Andreas K. Hüttel 03a0041dcd
Upsync binpackages based on checksum
Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>
2024-03-14 11:21:43 +01:00
Ian Jordan 456173c321
tools/catalyst-auto: Remove treeish sed
Removes line which allowed catalyst-auto to convert to snapshot_treeish
on the fly as it's no longer needed.

Signed-off-by: Ian Jordan <immoloism@gmail.com>
Signed-off-by: Matt Turner <mattst88@gentoo.org>
2024-02-20 15:43:53 -05:00
Andreas K. Hüttel be2fa5fd67
Write some stats about binpkg uploads in the log
Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>
2024-02-05 22:12:39 +01:00
Andreas K. Hüttel 47eb54bb04
catalyst-auto: do not upload zero-size binpackage files
Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>
2023-12-07 23:10:48 +01:00
Andreas K. Hüttel 2f84a3644d
Fix precise rsync syntax
Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>
2023-08-06 00:41:54 +02:00
Andreas K. Hüttel 3bc092d566
catalyst-auto: add function upsync_binpackages
Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>
2023-08-05 21:06:23 +02:00
Andreas K. Hüttel e0097a27a9
Drop quotes around variable strings
Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>
2023-03-05 23:42:46 +01:00
Nic Boet fe7ecde00f
Do not source catalyst.conf as it's not bash syntax
* non-bash formatted lines are executed; results in several "command not found" errors
* instead, parse the file for the specific var

[this still leaves the doublequotes " around the value- dilfridge]

Signed-off-by: Nic Boet <nic@boet.cc>
Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>
2023-03-05 23:38:17 +01:00
Nic Boet 249185d427
snapshot_cache was removed from catalyst Apr 2020
* catalyst commit 6493f3bc534f97468c514a12035b10e3107c4ecf
* --preclean would now fail as catalyst.conf is invalid

Signed-off-by: Nic Boet <nic@boet.cc>
Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>
2023-03-02 19:28:48 +01:00
Nic Boet bdebb3c30e
Improve send_mail() because it acts more like the logger
* output failure events to stdout if verbose set, useful for those without a mailer
 * do not email if recipient var is undef; allows flexibility for non-releng use cases

Signed-off-by: Nic Boet <nic@boet.cc>
Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>
2023-02-27 00:27:02 +01:00
Nic Boet f2e3c07e81
Make it more apparent that bash time is not used
Signed-off-by: Nic Boet <nic@boet.cc>
Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>
2023-02-27 00:18:54 +01:00
Andreas K. Hüttel d5223d3408
catalyst-auto: e-mail more lines of failed logs (1000 instead of 200)
It helps if the log contains the actual error

Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>
2022-07-18 20:39:54 +02:00
Matt Turner ba25e990d0
tools: Upload with SSH option IPQoS=cs0
SSH defaults to IPQoS=cs1 for non-interative sessions, like scp.
OSUOSL's network prioritizes these packets lower and often drops them,
reducing uplaod performance significantly.

Signed-off-by: Matt Turner <mattst88@gentoo.org>
2021-08-26 12:54:52 -07:00
Matt Turner 7b51adc752
tools: Simplify nonetwork handling
... and fix the indentation.

Signed-off-by: Matt Turner <mattst88@gentoo.org>
2021-01-31 18:27:13 -05:00
Matt Turner 174254bbb1
tools: Include post_build.log contents when successful
Signed-off-by: Matt Turner <mattst88@gentoo.org>
2021-01-31 18:22:07 -05:00
Matt Turner 7ec486fab4 tools: Consolidate upload() function
Signed-off-by: Matt Turner <mattst88@gentoo.org>
2021-01-30 20:57:12 -05:00
Andreas K. Huettel 0c7067713b
Add catalyst-auto option --nonetwork
Signed-off-by: Andreas K. Huettel <dilfridge@gentoo.org>
2021-01-08 22:09:03 +02:00
Matt Turner e02f16792b tools: Skip copying nonexistent kconfigs
Signed-off-by: Matt Turner <mattst88@gentoo.org>
2020-12-17 17:20:31 -05:00
Andreas K. Huettel e2d27f67f5
catalyst-auto: Allow parallel building of spec sets
This assumes that the spec sets are logically independent from each other,
i.e., building different ABIs.

Code shamelessly adapted from locale-gen.

Signed-off-by: Andreas K. Huettel <dilfridge@gentoo.org>
2020-08-29 15:33:37 +03:00
Mike Frysinger 0d1104d04f catalyst-auto: fix preclean logic after snapshot changes
The snapshot generation logic was moved earlier in the build (before
the cleaning logic) which caused the preclean to nuke the freshly
generated cache.  Move all the preclean logic back up before that.

Signed-off-by: Mike Frysinger <vapier@gentoo.org>
2020-05-31 12:43:59 -04:00
Mike Frysinger cfc55d1712 catalyst-auto: standardize indentation
This file uses tabs in most of it, so migrate the few funcs using
spaces over to tabs.

Signed-off-by: Mike Frysinger <vapier@gentoo.org>
2020-05-31 12:32:02 -04:00
Matt Turner a09f3dfda4 catalyst-auto: Don't send useless failure email
Signed-off-by: Matt Turner <mattst88@gentoo.org>
2020-05-17 15:29:53 -07:00
Robin H. Johnson 703d2ef0c1
catalyst-auto: print full path to specfile for the email
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
2020-05-11 22:47:03 -07:00
Robin H. Johnson 62bfd232bb
catalyst-auto: defer purge until all sets are done
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
2020-05-11 22:46:34 -07:00
Ben Kohler 05592ee0b7
catalyst-auto: set DATESTAMP again
This var is still used by some arches' catalyst-auto configs

Signed-off-by: Ben Kohler <bkohler@gentoo.org>
2020-05-11 13:53:55 -05:00
Robin H. Johnson 77b29c8686
tools/catalyst-auto: permit OTHER flags in /etc/resolv.conf
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
2020-05-07 21:59:19 -07:00
Matt Turner 51d806875e catalyst-auto: Add fallback for unset storedir
Signed-off-by: Matt Turner <mattst88@gentoo.org>
2020-04-30 16:12:56 -07:00
Matt Turner 943e1a7181 catalyst-auto: Support sqfs snapshots
Signed-off-by: Matt Turner <mattst88@gentoo.org>
2020-04-29 23:04:59 -07:00
Matt Turner 43aa276417 catalyst-auto: Drop unnecessary spec munging
No longer necessary with @TIMESTAMP@.

Signed-off-by: Matt Turner <mattst88@gentoo.org>
2020-04-29 12:37:50 -07:00
Matt Turner 369e27e399 catalyst-auto: Drop setting @DATESTAMP@
Not used in any of the spec files.

Signed-off-by: Matt Turner <mattst88@gentoo.org>
2020-04-28 21:36:06 -07:00
Matt Turner e24e331fa3 catalyst-auto: Ensure 'options edns0' is in /etc/resolv.conf
Signed-off-by: Matt Turner <mattst88@gentoo.org>
2020-04-25 22:23:48 -07:00
Matt Turner e9ea15ab8b Upload stages to releng-incoming.gentoo.org
Most architecture's build systems are hosted at OSUOSL. The place the
stages need to end up is ultimately masterreleases.gentoo.org, also at
OSUOSL. For some unknown reason, instead of rsync'ing from one system to
another in the same building, we were sending the stages first (slowly,
I might add) to nightheron in France, before sending them back to
OSUOSL.

robbat2 has added a releng-incoming.gentoo.org DNS record that currently
points to nightheron. This will allow us to switch the record in the
future and save the stages from needing to circumnavigate the globe
before reaching the mirrors.

Signed-off-by: Matt Turner <mattst88@gentoo.org>
2020-04-25 22:18:34 -07:00
Matt Turner 492aacba77 catalyst-auto: Combine common functions
Signed-off-by: Matt Turner <mattst88@gentoo.org>
2019-09-01 19:04:59 -07:00
Mike Frysinger c63938c5e0 catalyst-auto: run git merge in quiet mode 2018-01-16 11:24:43 -05:00
Mike Frysinger bbca329576 catalyst-auto: log the command in the log file itself
This should make it clear in each log file what is being run, especially
when a phase is run more than once (like post_build).
2018-01-15 22:42:07 -05:00
Mike Frysinger f98bc791e6 catalyst-auto: pass current targets down to post_build
Commit b7e493b4e4 (run trigger_post_build
after every successful spec) broke a lot of builders.  The configs were
written to be called only after everything was done, but by calling it
after the first build (stage1 only), all the other artifacts we expected
did not yet exist.

Change post_build to pass the current set & spec name down so the config
can figure out what is available at that point in time.
2018-01-15 20:52:14 -05:00
Mike Frysinger 8f6370f8b3 catalyst-auto: run update_symlinks by default in pre_build
All configs defining update_symlinks run it in pre_build, and all the
other configs will want to do the same.  So make it the default in the
pre_build phase.
2018-01-15 20:42:19 -05:00
Mike Frysinger a8e70e1976 catalyst-auto: move git update into main script
All of our config files are doing this already, so move it into the
common code.
2018-01-15 20:17:43 -05:00
Mike Frysinger 04b56eaab3 catalyst-auto: move container logic into a func too
This lets us put it after command line parsing.
2018-01-15 20:13:59 -05:00
Mike Frysinger b291ea4754 catalyst-auto: move all code into functions
Mixing inline funcs and executable code makes it hard to follow and
shuffle ordering of operations.  Put everything other than variables
into functions.

The delta here is large, but it's almost entirely indentation changes.

This also makes updating the script (like `git pull`) more resilient.
2018-01-15 20:11:44 -05:00
Robin H. Johnson 5d73c6b04a
catalyst-auto: add intermediate verbose, that prints the command, but not the output; -vv has old behavior
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
2017-12-28 21:32:10 -08:00
Robin H. Johnson af9a181c70
catalyst-auto: fix title.
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
2017-12-15 10:40:07 -08:00
Robin H. Johnson 85bac3a8a2
catalyst-auto: only purge tmpdir+autoresume, not pkgdir.
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
2017-12-14 19:45:09 -08:00
Robin H. Johnson b7e493b4e4
catalyst-auto: run trigger_post_build after every successful spec, to export results sooner.
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
2017-12-14 19:44:41 -08:00
Robin H. Johnson 561af1245f
catalyst-auto/run_cmd: append logs instead of overwrite
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
2017-12-14 19:44:07 -08:00
Robin H. Johnson ed258c751b
tools/catalyst-auto: change datestamp->timestamp
Prepare for doing multiple automated runs in a single day on
newer/faster build systems.

Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
2017-12-07 23:40:38 -08:00
Robin H. Johnson 44b56bd2f8
catalyst: do not use --purge so we can hit binpkgcache.
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
2017-12-06 23:01:35 -08:00
Mike Frysinger 1de39bb16d catalyst-auto: use full system hostname by default 2017-03-22 22:14:42 -04:00