102 Commits

Author SHA1 Message Date
Christoph Reiter
9e2e1814a2 Fix the failed run ID URL; use a .txt extension for the failed files 2021-01-11 20:46:48 +01:00
Christoph Reiter
491f3f9924 flake8 fixes 2021-01-09 14:46:13 +01:00
Naveen M K
5541c688e7
Add actions job run URL in staging-failed (#22) 2021-01-09 13:06:19 +01:00
Christoph Reiter
dae62d58bf Skip some more packages
blender takes too long
2021-01-04 16:53:45 +01:00
Christoph Reiter
b9b725c2ef
Merge pull request #21 from jeremyd2019/patch-1
add --pretend option to fetch-assets
2021-01-03 21:57:38 +01:00
Christoph Reiter
f36ff3d808 check if any blocked, not just the last repo 2021-01-03 21:55:04 +01:00
Christoph Reiter
de1805c9f0 Print the correct block list 2021-01-03 21:53:05 +01:00
Christoph Reiter
828d85c167 skip one more 2021-01-03 21:52:56 +01:00
Christoph Reiter
e3869cacf0 Skip some packages when checking if all rdeps of packages are ready
These are things that are unlikely to be fixed in the comming days,
so ignore them for now and don't block everything because of them.
2021-01-03 21:50:33 +01:00
Christoph Reiter
37e9074bb9 Fix typo 2021-01-03 21:42:21 +01:00
jeremyd2019
8d6d192a50
add --pretend option to fetch-assets
Sometimes I just want to see what would (and would not) be downloaded, but not actually download anything.
2021-01-03 12:40:25 -08:00
Christoph Reiter
bd99faf213 fetch-assets: ignore msys packages when trying to figure out if mingw ones can be downloaded 2020-12-31 09:28:16 +01:00
Christoph Reiter
6dcba93e99 fetch-assets: add --verbose and --fetch-all options 2020-12-31 09:07:29 +01:00
Christoph Reiter
2972d604ae oops.. 2020-12-29 11:46:17 +01:00
Christoph Reiter
57d6718655 Revert "Make ext-depends a set, now that packages are hashable"
This reverts commit cdd07772e74f9b2627ac1321e67d8b79bea09fc0.

We still need the name of the real package... so this was wrong
2020-12-29 11:39:00 +01:00
Christoph Reiter
4911712d46 Rework fetch-assets blocking
Instead of grouping the packages, just look at the deps and rdeps of each package,
this is actually what we want and easier.
2020-12-29 11:22:37 +01:00
Christoph Reiter
cdd07772e7 Make ext-depends a set, now that packages are hashable 2020-12-29 09:44:36 +01:00
Christoph Reiter
8e3ed38f5c get_packages_to_build: allow without creds 2020-12-28 08:58:38 +01:00
Christoph Reiter
03194d7f8b fetch-assets: be more strict in what we allow to download
In case any builds are in the queue that depend on finished builds in the
queue don't download both.

In case finished builds are in the queue and new dependencies get added
to the queue don't download both.

Handle this by grouping into separate dependency graphs and only download
finished sub graphs.

Might need a bit more feedback as to what and why it is skipped.

Fixes #19
2020-12-26 19:28:59 +01:00
Christoph Reiter
ca2a75da4c Only warn when no creds are given 2020-12-26 17:17:08 +01:00
Christoph Reiter
dfb80bed37 Allow calling fetch_assets without credentials
It will only work a few times, but it's usually only used once anyway.

Fixes #20
2020-12-15 21:54:35 +01:00
Christoph Reiter
58febc1140 switch back to keys.gnupg.net 2020-12-11 16:05:06 +01:00
Christoph Reiter
c220de72ed Don't fail when a package gets updated during a build
If there is a new version it's not a duplicate
2020-12-08 09:30:58 +01:00
Christoph Reiter
00f7657c4d fetch-assets: chmod the downloaded files to 644
tempfile creates them with 600 by default, set back to 644, which
is what it was before the tempfile change.
2020-12-07 20:59:31 +01:00
Christoph Reiter
0c0d36fbdc fetch_assets: use more threads and increase the read timeout a bit
bumping the thread count makes things a bit faster here, and to not hit
a read timeout, increase that a bit as well.
2020-12-01 19:25:19 +01:00
Christoph Reiter
68a4ac86cc download_assets: download to a temp file and then move to the destination
In case the download fails we don't want to end up with half downloaded
files in the target location.
2020-12-01 19:25:18 +01:00
Christoph Reiter
599a5db844 Handle all cases when deciding wether to build source packages
It only looked at the mingw64 build to see if a source package should
be built. But in can also be that only a mingw32 is needed if the other
one was already updated.
2020-11-22 18:17:17 +01:00
Christoph Reiter
9ae4de6fcb Handle assets that failed to upload completely
For deciding what to build we need to ignore them and also
replace them if needed. For cleaning things up we need to take
of them too.
2020-11-13 11:47:46 +01:00
Christoph Reiter
6e7d6a99ce Check binary deps when testing if a source package can be build 2020-10-25 16:59:04 +01:00
Christoph Reiter
1a8409ab8a Fail right away if a dependency is missing
We can't continue anyway and it will just try again.
2020-10-25 16:13:29 +01:00
Christoph Reiter
3960ab29d1 Remove some dead code 2020-10-24 16:46:44 +02:00
Christoph Reiter
78a2861ae8 trigger: port to new workflow dispatch
this is now exposed in pygithub, allows us to trigger the same way
as with the website UI.
2020-10-23 11:18:20 +02:00
Christoph Reiter
9feb10b304 Simplify timeout handling
We no longer need to prevent the workflow hitting the limit since we
handle partial builds now.
2020-10-23 10:09:49 +02:00
Christoph Reiter
fe669e0bf1 Skip the workflow if there is already one running 2020-10-23 09:48:36 +02:00
Christoph Reiter
0ef0f21ff0 Pass lazy to get_repo()
fewer requests hopefully
2020-10-22 22:17:49 +02:00
Christoph Reiter
34e56d5e5d Set the per page count to the maximum, from 30 to 100
larger pages -> fewer requests -> less likely to hit api rate limit
2020-10-22 22:10:14 +02:00
Christoph Reiter
c7f6e58483 Try to reduce the amount of API requests
We are hitting the rate limit, try to keep objects around and reuse them.
2020-10-13 20:07:18 +02:00
Christoph Reiter
ef81fcc381 Remember skipped states for later packages
if A is skipped and C -> B -> A, then C should be skipped too.
2020-10-08 12:23:04 +02:00
Christoph Reiter
63063cbc8e
Skip mingw-w64-qt5-static
Takes too long
2020-10-08 08:57:48 +02:00
Christoph Reiter
a0c4802fdb Pull all packages from staging that got built together
It could be that the build results depen on each other with a fixed version.
We need to add them all to the local repo for the install/update to work.
2020-09-21 19:17:53 +02:00
Christoph Reiter
d8498e2e7f
skip mingw-w64-firebird-git
takes too long
2020-09-21 06:42:59 +02:00
Christoph Reiter
0b94083866
skip mingw-w64-gcc-git 2020-09-20 17:55:45 +02:00
Christoph Reiter
94a242b27f Include vcs packages again
we should be handling this "ok" now. If the build produces a different version
than expected we will mark the build as failed.
2020-09-20 16:38:08 +02:00
Christoph Reiter
48bde21983 another shadowed var 2020-09-20 10:26:58 +02:00
Christoph Reiter
2aaaaafffd Skip src builds even if none of the binary packages have been built 2020-09-19 20:49:13 +02:00
Christoph Reiter
425762d6b3 fetch-assets: only download files where everything for the package has been built
to avoid us uploading partial builds
2020-09-19 20:43:18 +02:00
Christoph Reiter
3e33c2a21a Disable the manual build timeout 2020-09-19 08:58:59 +02:00
Christoph Reiter
94c51cb2c0 show-assets: ignore files that need to be deleted 2020-09-18 11:14:54 +02:00
Christoph Reiter
9d53e3dd7c fetch-assets: don't download files which are in the repo
Instead of depending on the cleanup job just take it into account
when downloading the files.
2020-09-18 11:11:56 +02:00
Christoph Reiter
2ca668b8e6 mypy fixes 2020-09-18 11:11:43 +02:00