456 Commits

Author SHA1 Message Date
Christoph Reiter
0d25d51a04 Convert the script to a Python package
It can now be invoked via `python -m msys2_autobuild` or
by installing it, which adds a "msys2-autobuild" script.

This is a first step towards splitting up the code.

The HTTP cache is now stored in the working directory
instead of the source directory.
2023-03-21 11:34:39 +01:00
Christoph Reiter
d0ddf60737 Update dependencies 2023-03-18 10:40:06 +01:00
Christoph Reiter
91ab34350f cache: clean up at the end and limit to 3 hours
it's unlikely there will be many hits after some hours, so better
keep the upload size low. Also clean at the end to make
the upload smaller.
2023-02-19 17:02:55 +01:00
Christoph Reiter
38e6bc6e47 requests_cache: port to new cache cleanup function
I find the API still confusing, but it's better then before.
2023-02-19 16:43:35 +01:00
Christoph Reiter
6ccea00bba Bump the max number of jobs again
Since the last commit we should need fewer API calls
2023-02-19 16:10:29 +01:00
Christoph Reiter
c152a6dbbf Depend on the new pygithub assets API
This exposes the assets inline from a release, so this
should save us lots of requests. Available since v1.58.0
2023-02-19 16:08:51 +01:00
Christoph Reiter
b7df29ff56 CI: skip installing wheel
This was for packages without wheels to build them initially.
In theory newer pip should handle this automatically, let's see
2023-02-19 16:07:19 +01:00
Christoph Reiter
77c2d02a4d Update dependencies 2023-02-19 16:03:41 +01:00
Jeremy Drake
aea263ec2c CI: Remove enabling of clangarm64 in pacman.conf
It is now enabled by default so this is a no-op.
2023-01-29 00:33:11 +01:00
Jeremy Drake
1666f6d3b0 Allow building qt6-static on clangarm64.
The timeout on a self-hosted runner is much larger (72 hours, though
there seemed to be a different limit related to the token hit before
reaching that).
2023-01-18 19:44:42 +01:00
Christoph Reiter
63a1b6020e Extend manual build for mingw-w64-qt5-static to clang32/64
seems the clang build got slower, it now hits the 6h limit always
2023-01-18 18:36:19 +01:00
Christoph Reiter
77a21114a8 CI: set "MSYS" env var later
so the cache action doesn't override it
See https://github.com/actions/toolkit/issues/1312
2023-01-14 12:33:26 +01:00
Christoph Reiter
6e2c5b47d4 Revert "Unset MSYS env everywhere"
This reverts commit e2f4f874a20304bc94047ddf92ca63a9ee9aa5e5.

We are depending on it being set in CI, so this isn't the right approach
2023-01-14 12:32:40 +01:00
Christoph Reiter
e2f4f874a2 Unset MSYS env everywhere
See https://github.com/actions/toolkit/issues/1311
2023-01-14 07:58:13 +01:00
Christoph Reiter
63f65d30bc Delete old assets in a thread pool
To speed things up a bit
2023-01-01 11:34:17 +01:00
Christoph Reiter
307799fd27 Update the status file format and include cycles
This moves it closer to the buildqueue format, and also includes cycles,
and allows future additions.
2022-12-27 16:16:47 +01:00
Christoph Reiter
bf82f9fff2 Don't include broken cycles in the cycle output 2022-12-27 16:16:08 +01:00
Christoph Reiter
a9862b27c1 Missed one left over src build-type 2022-12-24 00:03:29 +01:00
Christoph Reiter
2ae439cd00 Build all source packages in a separate build job
See https://github.com/msys2/msys2-autobuild/issues/69

Building source packages requires git etc to be installed, but
ideally we wouldn't pollute the builder with extra packages that
it doesn't explicitly require.

To avoid this build msys and mingw source packages in a separate job.
2022-12-23 23:53:08 +01:00
Christoph Reiter
21a84297d8 Update deps 2022-12-21 12:09:05 +01:00
Christoph Reiter
e22cc1cc17 Update dependencies 2022-12-10 21:55:46 +01:00
Christoph Reiter
eee25ec33f CI: run on ubuntu-22.04 2022-12-10 21:21:01 +01:00
Christoph Reiter
59e8e1af5d CI: create a larger pagefile
so we can build flang in CI, same as https://github.com/msys2/MINGW-packages/pull/13791
2022-10-29 21:18:54 +02:00
Christoph Reiter
1fd41adbfa CI: test with 3.11 2022-10-27 08:05:20 +02:00
Christoph Reiter
e94b92f73e Update deps 2022-10-27 08:04:11 +02:00
Christoph Reiter
5d06444a57 CI: port away from ::set-output 2022-10-21 13:24:22 +02:00
Christoph Reiter
9d582e19b1 Build src packages in an ucrt64 env
It will be the new default
2022-10-10 18:39:24 +02:00
Christoph Reiter
bf34129d62 Update dependencies 2022-10-09 20:57:44 +02:00
Christoph Reiter
c9dd9afe5e Unset VCPKG_ROOT during build
see https://github.com/msys2/MINGW-packages/pull/13368
2022-10-02 12:35:50 +02:00
Christoph Reiter
b40229daa6 Drop BUILD_TYPES_WIP
This wasn't complete as it would only ignore broken builds
for direct deps and not indirect ones, but kinda worked in ignoring
some arm64 errors.

But it also causes problems if an error is ignored and the other arches
get uploaded. Then it's hard to roll back the update because lots of
packages with the new version are already in the repo.

With the new autobuild controller we can also restart flaky builds instead
of ignoring them and waiting for jeremy to fix them later.

Let's try removing that special case.
2022-09-20 08:02:56 +02:00
Jeremy Drake
253f8b8c4c GHA: accept extra 'context' input
This is meant for the invoker (ie, msys2-autobuild-controller) to
provide additional information to be logged with the job (specifically,
what user requested it).
2022-09-06 21:58:16 +02:00
Jeremy Drake
c03c642719 GHA: log workflow_dispatch inputs in job 2022-09-06 21:58:16 +02:00
Christoph Reiter
f581199930 try running the real pacman with exec
it seems like the pacman wrapper doesn't survive a runtime update.
try exec to avoid returning control to bash
2022-09-06 21:11:16 +02:00
Christoph Reiter
3637fea711 Update dependencies 2022-09-04 10:33:47 +02:00
Christoph Reiter
e23492ee15 also retry on 502
We just got "502 {"message": "Server Error"}" on a DELETE
2022-09-02 21:53:46 +02:00
Christoph Reiter
9f4f288d00 retry HTTP requests which return 500
We are getting "500 null" randomly recently, mabye this helps.
2022-08-27 13:51:32 +02:00
Christoph Reiter
b36a4da1da Use a temporary pacman.conf during building
Up until now we created a backup of pacman.conf and restored it after
the build was done. This can leave the environment in an undefined state
if something crashes inbetween.

Instead create a temporary pacman.conf and use that during building.
In theory pacman allows setting a custom config via "--config", but
makepkg doesn't expose this, so that's not working. Luckily makepkg
allows overriding the pacman path via the PACMAN env var, so we create
a temporary script which just forwards everything to pacman and always
sets the temporary config.
2022-08-26 15:09:05 +02:00
Christoph Reiter
5ecdbc97a7 thinko
we are in bash here, not powershell...
2022-08-21 21:52:59 +02:00
Christoph Reiter
0d4680c01f Don't use workflow inputs directly in scripts
They could inject commands that way. Instead assign them
to an env var and then use that env var in the powershell scripts.

We want to open those controls up to more people, so we need to make
sure they can only change the values and not extract tokens etc.

Fixes #60
2022-08-21 20:37:07 +02:00
Christoph Reiter
9374b1d9b4 main: run update-status at the end 2022-08-15 14:03:50 +02:00
Jeremy Drake
22ea970beb CI: uncomment clangarm64 from pacman.conf.
instead of adding from scratch.  Once the commented-out section was
added, the grep would match that and no longer run the sed to add it.

Also remove line adding clang32 section because that was added to
default pacman.conf (and was thus a no-op).
2022-08-14 22:39:52 +02:00
Christoph Reiter
45c6b89ec7 Update the build status before stopping due to timeout
In case the job stops because it has reached the time limit it would
not update the build status and just quit. Move the timeout check
a bit later to acoid that.
2022-08-08 18:50:27 +02:00
Christoph Reiter
70c6903191 CI: update python to 3.10 and setup-python to v4 2022-08-04 21:45:15 +02:00
Christoph Reiter
f33be41b0f Update dependencies 2022-08-04 21:39:19 +02:00
Christoph Reiter
5f53dab6de Enable winjitdebug to workaround python crashing issues
Why this helps, I don't know..
2022-07-24 12:31:25 +02:00
Christoph Reiter
4dbd2618fb Update deps 2022-07-21 21:54:07 +02:00
Christoph Reiter
a43bdf9479 Add a comment as to why we split up repo-add calls
This was pointed out here: 7d84a7e086 (r75916830)
2022-06-30 21:07:00 +02:00
Christoph Reiter
9360a8eebe Update dependencies 2022-06-30 21:04:54 +02:00
Christoph Reiter
7d84a7e086 Limit the amount of packages added with repo-add in one go
It errors out if there are too many (maybe memory?)
2022-06-12 18:28:48 +02:00
Christoph Reiter
ea46306e71 Add config key for limiting the max job count
We are hitting the API limit again, so reduce from 15 to 12.
This also allows self hosted runners to limit to 1 if needed.
2022-06-11 18:32:11 +02:00