Commit graph

132 commits

Author SHA1 Message Date
Ben Kohler f7891ac366
tools/catalyst-auto-{amd64,x86}.conf: fix update_symlinks globbing
Signed-off-by: Ben Kohler <bkohler@gentoo.org>
2020-08-12 10:43:54 -05:00
Ben Kohler 6fbf103e56
cataylst-auto-amd64.conf: disable stage4 builds
No longer needed, per prometheanfire

Signed-off-by: Ben Kohler <bkohler@gentoo.org>
2020-07-06 16:48:03 -05:00
Ben Kohler 6e5eb93fb3
catalyst-auto-amd64.conf: re-enable selinux builds
Signed-off-by: Ben Kohler <bkohler@gentoo.org>
2020-06-30 05:38:47 -05:00
Ben Kohler c0ebcf37c7
catalyst-auto-amd64.conf: missing amd64- in systemd stage3 filenames
Signed-off-by: Ben Kohler <bkohler@gentoo.org>
2020-05-25 06:09:56 -05:00
Matt Turner a24359b95c amd64: Add systemd specs
Signed-off-by: Matt Turner <mattst88@gentoo.org>
2020-05-21 18:46:51 -07:00
Ben Kohler a1f347da83
catalyst-auto-amd64.conf: disable selinux builds for now
Due to https://bugs.gentoo.org/722108 , selinux stable builds cannot
possibly succeed right now.  We are wasting lots of hours in update_seed
since that part cannot be cached.  We can re-enable them when
https://bugs.gentoo.org/714376 is fixed (stabilization).

Signed-off-by: Ben Kohler <bkohler@gentoo.org>
2020-05-20 08:52:43 -05:00
Ben Kohler ef26f9a26f
catalyst-auto-{amd64,x86}*.conf: add info echo to post_build
Added a small debug echo so that post_build.log shows some information
about what's being uploaded.  Might add destination info later.

Signed-off-by: Ben Kohler <bkohler@gentoo.org>
2020-05-17 10:31:07 -05:00
Ben Kohler 60a1326315
catalyst-auto-{amd64,x86}*.conf: change cmd array to upload function
Use an easier-to-read "upload" function in place of "${cmd[@]}" array
syntax.  Added a helper var UPLOAD_DEST to be set in advance so that
upload function takes only a list of filenames to be uploaded.

Signed-off-by: Ben Kohler <bkohler@gentoo.org>
2020-05-15 12:34:12 -05:00
Ben Kohler 915dc435ff
catalyst-auto-amd64*.conf: explicitly list files to be transferred
Previously when we were copying locally there was no (major) problem
with wildcarding a long list of files to be transferred after every
single spec finishes.  Now that we are transferring to a remote host,
this results in a lot of wasted transfer time & bandwidth.

I have adjusted the post_build function to only transfer the new files
known to be produced by each spec (which has files meant to be
published).

Signed-off-by: Ben Kohler <bkohler@gentoo.org>
2020-05-15 07:40:02 -05:00
Robin H. Johnson affe649205
catalyst-auto: no special behavior for nightheron copy anymore
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
2020-05-11 22:14:35 -07:00
Matt Turner 8407af57f5 Remove weekly directory
Signed-off-by: Matt Turner <mattst88@gentoo.org>
2020-04-28 18:18:02 -07:00
Matt Turner e9ea15ab8b Upload stages to releng-incoming.gentoo.org
Most architecture's build systems are hosted at OSUOSL. The place the
stages need to end up is ultimately masterreleases.gentoo.org, also at
OSUOSL. For some unknown reason, instead of rsync'ing from one system to
another in the same building, we were sending the stages first (slowly,
I might add) to nightheron in France, before sending them back to
OSUOSL.

robbat2 has added a releng-incoming.gentoo.org DNS record that currently
points to nightheron. This will allow us to switch the record in the
future and save the stages from needing to circumnavigate the globe
before reaching the mirrors.

Signed-off-by: Matt Turner <mattst88@gentoo.org>
2020-04-25 22:18:34 -07:00
Matt Turner 8f5cec8809 Revert "Upload stages to releng-incoming.gentoo.org"
This causes an interactive prompt:

    Are you sure you want to continue connecting (yes/no/[fingerprint])?

The plan is to get the SSH fingerprint via DNSSEC, so I'll revert this
change until that is in place and wired up.

This reverts commit 63174346f2.

Signed-off-by: Matt Turner <mattst88@gentoo.org>
2020-04-23 14:41:13 -07:00
Matt Turner 63174346f2 Upload stages to releng-incoming.gentoo.org
Most architecture's build systems are hosted at OSUOSL. The place the
stages need to end up is ultimately masterreleases.gentoo.org, also at
OSUOSL. For some unknown reason, instead of rsync'ing from one system to
another in the same building, we were sending the stages first (slowly,
I might add) to nightheron in France, before sending them back to
OSUOSL.

robbat2 has added a releng-incoming.gentoo.org DNS record that currently
points to nightheron. This will allow us to switch the record in the
future and save the stages from needing to circumnavigate the globe
before reaching the mirrors.

Signed-off-by: Matt Turner <mattst88@gentoo.org>
2020-04-23 00:14:11 -07:00
Matt Turner 492aacba77 catalyst-auto: Combine common functions
Signed-off-by: Matt Turner <mattst88@gentoo.org>
2019-09-01 19:04:59 -07:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) ce743c3886 Stop building the admincd for selinux.
Signed-off-by: Jorge Manuel B. S. Vicetto (jmbsvicetto) <jmbsvicetto@gentoo.org>
2019-06-07 00:37:34 +00:00
Matthew Thode b7ec6e6805
remove systemd cruft 2018-03-02 10:19:08 -06:00
Matthew Thode 71795cbe91
Revert "enable systemd stage generation"
This reverts commit 60d2a138de.

using blueness's run-systemd stuff
2018-03-02 09:04:04 -06:00
Matthew Thode dda62ba027
Revert "add systemd to sets"
This reverts commit f249d95649.

using blueness's run-systemd stuff
2018-03-02 09:03:25 -06:00
Matthew Thode f249d95649
add systemd to sets 2018-03-02 06:36:07 -06:00
Matthew Thode 60d2a138de
enable systemd stage generation 2018-03-02 06:32:59 -06:00
Matthew Thode 57a5bfeb2a
Revert "move stage4 build integration stages into a subdir"
This reverts commit 11ccab53f8.
2018-02-04 18:21:25 -06:00
Matthew Thode 11ccab53f8
move stage4 build integration stages into a subdir 2018-02-04 17:06:42 -06:00
Mike Frysinger 8f6370f8b3 catalyst-auto: run update_symlinks by default in pre_build
All configs defining update_symlinks run it in pre_build, and all the
other configs will want to do the same.  So make it the default in the
pre_build phase.
2018-01-15 20:42:19 -05:00
Mike Frysinger a8e70e1976 catalyst-auto: move git update into main script
All of our config files are doing this already, so move it into the
common code.
2018-01-15 20:17:43 -05:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 93bc214182 Sync amd64 and x86 base and experimental scripts. Add a few missing popd calls to base scripts.
Signed-off-by: Jorge Manuel B. S. Vicetto (jmbsvicetto) <jmbsvicetto@gentoo.org>
2017-12-27 01:54:39 -01:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 0c3d971bfb Fix typo.
Signed-off-by: Jorge Manuel B. S. Vicetto (jmbsvicetto) <jmbsvicetto@gentoo.org>
2017-12-22 11:29:30 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 495425ec6a Drop old set definition.
Signed-off-by: Jorge Manuel B. S. Vicetto (jmbsvicetto) <jmbsvicetto@gentoo.org>
2017-12-22 11:22:26 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 3190fadba2 Use 2 vars for destinations (one for default and another for hardened).
Signed-off-by: Jorge Manuel B. S. Vicetto (jmbsvicetto) <jmbsvicetto@gentoo.org>
2017-12-22 11:02:44 +00:00
Robin H. Johnson 0292a7e2b5
tools/catalyst-auto-{x86,amd64}: refactor post_build for remote buildhost.
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
2017-12-14 21:48:20 -08:00
Robin H. Johnson d9f56a2c95
tools/catalyst-auto-amd64: stage4 builds should be optional.
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
2017-12-14 21:28:22 -08:00
Robin H. Johnson a5c5c7faad
catalyst-auto-amd64: verbosely update_symlinks.
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
2017-12-14 19:39:33 -08:00
Robin H. Johnson c19080d5f4
catalyst-auto: tweak timestamp.
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
2017-12-08 00:04:40 -08:00
Robin H. Johnson ed258c751b
tools/catalyst-auto: change datestamp->timestamp
Prepare for doing multiple automated runs in a single day on
newer/faster build systems.

Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
2017-12-07 23:40:38 -08:00
Matthew Thode c65a620930
remove stage4 cloud stages from tools/catalyst-auto-amd64.conf 2017-12-03 15:20:03 -06:00
Robin H. Johnson e243f99edf
catalyst-auto: squelch all pushd/popd
Have some sed magic, including a negative match.

git grep -l -e pushd -e popd \
| xargs sed -i -r \
  -e '/(pushd|popd)/{ /null/! { s,$, >/dev/null,g } }'

Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
2016-06-14 18:31:06 -07:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) bac6e3a2d7 Move the code to update symlinks to a separate function, call it from pre_build and after each set build.
Signed-off-by: Jorge Manuel B. S. Vicetto (jmbsvicetto) <jmbsvicetto@gentoo.org>
2016-05-05 10:07:20 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 11c84ac57d Whitespace. Stop hardcoding the bz2 extension and sync updates to the scripts to the experimental scripts.
Signed-off-by: Jorge Manuel B. S. Vicetto (jmbsvicetto) <jmbsvicetto@gentoo.org>
2016-04-26 22:35:19 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 39a3b94b39 Forgot to disable the systemd stage building.
Signed-off-by: Jorge Manuel B. S. Vicetto (jmbsvicetto) <jmbsvicetto@gentoo.org>
2016-03-13 22:46:14 -01:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 13528ad4e1 Add alternative systemd stages.
I'm going to use this in my build server. The official stages are being built by Anthony.

Signed-off-by: Jorge Manuel B. S. Vicetto (jmbsvicetto) <jmbsvicetto@gentoo.org>
2016-03-13 22:43:20 -01:00
Matthew Thode 06359b3c67
fix typo in catalyst config 2016-01-25 23:49:19 -06:00
Matthew Thode 56eaf4245f
enabling new minimal stage4 specs 2016-01-25 18:44:12 -06:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 656ca16378 Enable selinux stages.
Signed-off-by: Jorge Manuel B. S. Vicetto (jmbsvicetto) <jmbsvicetto@gentoo.org>
2016-01-13 21:12:39 -01:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) e7ded9f8dd Update catalyst-auto scripts for amd64 / x86 to add support for more targets and improve the detection / copy of stages.
Signed-off-by: Jorge Manuel B. S. Vicetto (jmbsvicetto) <jmbsvicetto@gentoo.org>
2015-12-27 02:37:12 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) bc954c38f2 Fix another spec name typo.
Signed-off-by: Jorge Manuel B. S. Vicetto (jmbsvicetto) <jmbsvicetto@gentoo.org>
2015-12-18 02:54:15 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 16bcda7b4e Fix stage4 spec name.
Signed-off-by: Jorge Manuel B. S. Vicetto (jmbsvicetto) <jmbsvicetto@gentoo.org>
2015-12-18 02:30:15 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 006fc1f047 Rename the stage4 to "cloud" in case we decide to add more stage4 later.
Signed-off-by: Jorge Manuel B. S. Vicetto (jmbsvicetto) <jmbsvicetto@gentoo.org>
2015-12-18 00:55:24 -01:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 29419b20d2 Add stage4 for Matthew's cloud images - https://github.com/prometheanfire/gentoo-cloud-prep/ .
Signed-off-by: Jorge Manuel B. S. Vicetto (jmbsvicetto) <jmbsvicetto@gentoo.org>
2015-12-16 01:58:22 -01:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 6dcdad447a Fix the circular latest symlinks by taking them out of the list of stages.
Sync amd64/x86 experimental script with base amd64/x86 script.
2013-08-16 01:23:03 +00:00
Raúl Porcel 585d8d645f Remove apostrophes 2013-08-14 17:23:42 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 5b98494124 Drop the date from the destination dir to avoid the double date directories in the mirrors. 2013-04-28 22:36:56 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) cf5df37ff0 Add the sets to create the selinux targets. They're not being added to sets, yet. 2013-04-20 15:21:31 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) bd55c0c987 Revert unintentional change to amd64 / x86 paths that is causing path weirdness on mirrors. 2013-04-18 08:28:12 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 666c0cdbe6 Build parent dirs if needed. 2012-10-14 04:30:56 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) f42a40df9a Move hardcoded paths out of the script files. 2012-10-13 04:06:36 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 8ee2ae6b0f Update config for amd64 / x86 to store stages under {default,hardened}. 2012-10-11 06:13:20 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 1c1dd9bf9e Update email address to skimmer and add nomultilib to the amd64 set. 2012-09-17 20:49:07 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 40a7d8dcce Update tools and config files for skimmer. 2012-09-17 01:21:57 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 25a5c10300 Add more checks and tests in postbuild for amd64 / x86. 2012-07-10 00:38:07 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) bd95b08675 Fix whitespace. 2012-06-27 23:58:05 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) c01d15caea Added x32 specs set and prepared the sets to build nomultilib and x32 targets. 2012-06-13 04:52:28 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) 8c70eae6ee Add the nomultilib set. The target won't be built until the set is added to sets. 2012-02-22 04:16:53 +00:00
Jorge Manuel B. S. Vicetto (jmbsvicetto) e38dced51f Update catalyst tools to use git for amd64 and x86. 2012-02-13 00:10:01 -01:00
Jorge Manuel B. S. Vicetto 0ad5787ec8 Add a few vars to the start of the script to reduce the hardcoded paths.
svn path=/trunk/; revision=812
2011-09-15 23:17:45 +00:00
Robin H. Johnson b477dc4322 Add new gentoo-releng-autobuilds list as a mail destination.
svn path=/trunk/; revision=771
2011-01-18 04:45:27 +00:00
Robin H. Johnson f24ecd86d0 Create a symlink for -latest as seed stage
Create a symlink for the latest name from the available stages3 always,
that is then used as the seed for building the new stages. To ensure
creation, always copy instead of move the stages to the destination
afterwards. In addition to this, we need a cleanup script, as before
only the isos and stages3 were moved, leaving the stage[12] eating up
space behind.

Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
Proposed-by: Jorge Manuel B. S. Vicetto <jmbsvicetto@gentoo.org>
Acked-by: Andrew Gaffney <agaffney@gentoo.org>

svn path=/trunk/; revision=769
2011-01-14 05:53:24 +00:00
Andrew Gaffney bb10ef43bb Add concept of build sets to catalyst-auto
svn path=/trunk/; revision=652
2009-06-27 19:09:30 +00:00
Andrew Gaffney aeebc831e3 remove period
svn path=/trunk/; revision=651
2009-05-26 19:05:11 +00:00
Andrew Gaffney 9121f66917 missing quote
svn path=/trunk/; revision=649
2009-05-14 02:53:27 +00:00
Andrew Gaffney 70b8826a72 move amd64 hardened builds into normal amd64 build
svn path=/trunk/; revision=644
2009-05-11 10:38:09 +00:00
Andrew Gaffney f5dd5097e0 Change post_build for all arches to only copy stage3/*.iso
svn path=/trunk/; revision=608
2009-01-02 22:27:47 +00:00
Andrew Gaffney fbc5444739 Add post_build() to move builds to /home/buildsync/
svn path=/trunk/; revision=595
2008-12-02 22:46:21 +00:00
Andrew Gaffney feff7ba1fa Add support for OPTIONAL_SPECS and purging
svn path=/trunk/; revision=591
2008-11-29 02:15:02 +00:00
Andrew Gaffney daa6f2bb89 Add x86 config and change email to releng@
svn path=/trunk/; revision=584
2008-10-26 16:26:16 +00:00
Andrew Gaffney df3cb8595a Use absolute path for svn checkout on poseidon
svn path=/trunk/; revision=583
2008-10-08 01:24:20 +00:00
Andrew Gaffney 5d28769332 Change sample config to point to new releases/weekly dir in checkout
svn path=/trunk/; revision=579
2008-09-28 19:40:48 +00:00
Andrew Gaffney 0b2162af44 basic pre_build()
svn path=/trunk/; revision=577
2008-09-28 04:43:08 +00:00
Andrew Gaffney 8b587404e2 Change a few config values
svn path=/trunk/; revision=574
2008-09-05 02:50:40 +00:00
Andrew Gaffney 117f098842 Copy kconfigs and mangle path in spec
svn path=/trunk/; revision=573
2008-09-04 12:44:33 +00:00
Andrew Gaffney 77d67f9f11 correct email address
svn path=/trunk/; revision=571
2008-09-04 02:32:01 +00:00
Andrew Gaffney 10dbf767ff Enable the build functionality
svn path=/trunk/; revision=570
2008-09-04 02:25:34 +00:00
Andrew Gaffney 85eeacd8cd Rename config
svn path=/trunk/; revision=569
2008-09-04 02:18:52 +00:00
Renamed from tools/catalyst-auto.conf (Browse further)