[cmake, tools, docs] Update CPMUtil (#2790)
updates CPMUtil to
[`8f9b39fc98`](8f9b39fc98)
Makes the tooling/docs better and enforces usage of tar.gz everywhere as
opposed to zips
Signed-off-by: crueter <crueter@eden-emu.dev>
Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/2790
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
Reviewed-by: MaranBr <maranbr@eden-emu.dev>
This commit is contained in:
crueter
parent
d7cd7c6313
commit
2e9dbe3f1d
30 changed files with 570 additions and 118 deletions
|
|
@ -2,7 +2,13 @@
|
|||
|
||||
These are supplemental shell scripts for CPMUtil aiming to ease maintenance burden for sanity checking, updates, prefetching, formatting, and standard operations done by these shell scripts, all in one common place.
|
||||
|
||||
All scripts are POSIX-compliant.
|
||||
All scripts are POSIX-compliant. If something doesn't work on your shell, ensure it's POSIX-compliant.
|
||||
* If your shell doesn't support `$(...)` syntax, you've got bigger problems to worry about.
|
||||
<!-- TOC -->
|
||||
- [Meta](#meta)
|
||||
- [Simple Utilities](#simple-utilities)
|
||||
- [Functional Utilities](#functional-utilities)
|
||||
<!-- /TOC -->
|
||||
|
||||
## Meta
|
||||
|
||||
|
|
@ -39,9 +45,8 @@ These scripts don't really have any functionality, they just help you out a bit
|
|||
* Inputs:
|
||||
- The repository (e.g. fmtlib/fmt)
|
||||
- The sha or tag (e.g. v1.0.1)
|
||||
- `GIT_HOST`: What git host to use (default github.com)
|
||||
- `USE_TAG`: Set to "true" if the second argument is a tag instead of a sha
|
||||
- `ARTIFACT`: The artifact to download, if using a tag. Set to null or empty to use the tag source archive instead
|
||||
- `-g <GIT_HOST>` or `--host <GIT_HOST>`: What git host to use (default github.com)
|
||||
- `-a <ARTIFACT>` or `--artifact <ARTIFACT>`: The artifact to download. Set to null or empty to use a source archive instead
|
||||
* Output: the SHA512 sum of the package
|
||||
- `url-hash.sh`: Determine the hash of a URL
|
||||
* Input: the URL
|
||||
|
|
@ -51,7 +56,7 @@ These scripts don't really have any functionality, they just help you out a bit
|
|||
|
||||
These modify the CPM cache or cpmfiles. Each allows you to input all the packages to act on, as well as a `<scriptname>-all.sh` that acts upon all available packages.
|
||||
|
||||
For the update and hash scripts, set `UPDATE=true` to update the cpmfile with the new version or hash. Beware: if the hash is `cf83e1357...` that means you got a 404 error!
|
||||
Beware: if a hash is `cf83e1357...` that means you got a 404 error!
|
||||
|
||||
- `fetch.sh`: Prefetch a package according to its cpmfile definition
|
||||
* Packages are fetched to the `.cache/cpm` directory by default, following the CPMUtil default.
|
||||
|
|
@ -61,11 +66,14 @@ For the update and hash scripts, set `UPDATE=true` to update the cpmfile with th
|
|||
- `check-updates.sh`: Check a package for available updates
|
||||
* This only applies to packages that utilize tags.
|
||||
* If the tag is a format string, the `git_version` is acted upon instead.
|
||||
* Setting `FORCE=true` will forcefully update every package and its hash, even if they are on the latest version (`UPDATE` must also be true)
|
||||
* Specifying `-f` or `--force` will forcefully update the package and its hash, even if it's on on the latest version.
|
||||
* Alternatively, only specify `-u` or `--update` to update packages that have new versions available.
|
||||
* This script generally runs fast.
|
||||
* Packages that should skip updates (e.g. older versions or packages with poorly-made tag structures... looking at you mbedtls) may specify `"skip_updates": true` in their cpmfile definition. This is unnecessary for untagged (e.g. sha or bare URL) packages.
|
||||
* Packages that should skip updates (e.g. older versions, OR packages with poorly-made tag structures... looking at you mbedtls) may specify `"skip_updates": true` in their cpmfile definition. This is unnecessary for untagged (e.g. sha or bare URL) packages.
|
||||
- `check-hashes.sh`: Check a package's hash
|
||||
* Specifying `-f` or `--force` will update the package's hash even if it's not mismatched.
|
||||
* Alternatively, specify `-u` or `--update` to only fix mismatched hashes.
|
||||
* This only applies to packages with hardcoded hashes, NOT ones that use hash URLs.
|
||||
* This script will take a looooooooooooooong time. This is operationally equivalent to a prefetch, and thus checking all hashes will take a while--but it's worth it! Just make sure you're not using dial-up.
|
||||
* This script will take a long time. This is operationally equivalent to a prefetch, and thus checking all hashes will take a while--but it's worth it! Just make sure you're not using dial-up.
|
||||
|
||||
You are recommended to run sanity hash checking for every pull request and commit, and weekly update checks.
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
#!/bin/bash -e
|
||||
#!/bin/sh -e
|
||||
|
||||
# SPDX-FileCopyrightText: 2025 crueter
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
|
@ -7,4 +7,4 @@
|
|||
. tools/cpm/common.sh
|
||||
|
||||
# shellcheck disable=SC2086
|
||||
tools/cpm/check-hash.sh $LIBS
|
||||
tools/cpm/check-hash.sh "$@" $LIBS
|
||||
|
|
@ -3,18 +3,49 @@
|
|||
# SPDX-FileCopyrightText: 2025 crueter
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
# env vars:
|
||||
# - UPDATE: fix hashes if needed
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
. tools/cpm/common.sh
|
||||
|
||||
RETURN=0
|
||||
|
||||
for PACKAGE in "$@"
|
||||
do
|
||||
usage() {
|
||||
cat << EOF
|
||||
Usage: $0 [uf] [PACKAGE]...
|
||||
Check the hash of a specific package or packages.
|
||||
If a hash mismatch occurs, this script will print the corrected hash of the package.
|
||||
|
||||
Options:
|
||||
-u, --update Correct the package's hash if it's a mismatch
|
||||
|
||||
-f, --force Update the package's hash anyways (implies -u)
|
||||
|
||||
Note that this procedure will usually take a long time
|
||||
depending on the number and size of dependencies.
|
||||
|
||||
This project has defined the following as valid cpmfiles:
|
||||
EOF
|
||||
|
||||
for file in $CPMFILES; do
|
||||
echo "- $file"
|
||||
done
|
||||
|
||||
exit $RETURN
|
||||
}
|
||||
|
||||
while true; do
|
||||
case "$1" in
|
||||
(-uf|-f|--force) UPDATE=true; FORCE=true; shift; continue ;;
|
||||
(-u|--update) UPDATE=true; shift; continue ;;
|
||||
(-h) usage ;;
|
||||
("$0") break ;;
|
||||
("") break ;;
|
||||
esac
|
||||
|
||||
PACKAGE="$1"
|
||||
|
||||
shift
|
||||
|
||||
export PACKAGE
|
||||
# shellcheck disable=SC1091
|
||||
. tools/cpm/package.sh
|
||||
|
||||
if [ "$CI" != null ]; then
|
||||
|
|
@ -31,11 +62,13 @@ do
|
|||
export USE_TAG=true
|
||||
ACTUAL=$(tools/cpm/url-hash.sh "$DOWNLOAD")
|
||||
|
||||
# shellcheck disable=SC2028
|
||||
[ "$ACTUAL" != "$HASH" ] && echo "-- * Expected $HASH" && echo "-- * Got $ACTUAL" && [ "$UPDATE" != "true" ] && RETURN=1
|
||||
if [ "$ACTUAL" != "$HASH" ]; then
|
||||
echo "-- * Expected $HASH"
|
||||
echo "-- * Got $ACTUAL"
|
||||
[ "$UPDATE" != "true" ] && RETURN=1
|
||||
fi
|
||||
|
||||
if [ "$UPDATE" = "true" ] && [ "$ACTUAL" != "$HASH" ]; then
|
||||
# shellcheck disable=SC2034
|
||||
if { [ "$UPDATE" = "true" ] && [ "$ACTUAL" != "$HASH" ]; } || [ "$FORCE" = "true" ]; then
|
||||
NEW_JSON=$(echo "$JSON" | jq ".hash = \"$ACTUAL\"")
|
||||
export NEW_JSON
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
#!/bin/bash -e
|
||||
#!/bin/sh -e
|
||||
|
||||
# SPDX-FileCopyrightText: 2025 crueter
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
|
@ -7,4 +7,4 @@
|
|||
. tools/cpm/common.sh
|
||||
|
||||
# shellcheck disable=SC2086
|
||||
tools/cpm/check-updates.sh $LIBS
|
||||
tools/cpm/check-updates.sh "$@" $LIBS
|
||||
|
|
@ -3,21 +3,51 @@
|
|||
# SPDX-FileCopyrightText: 2025 crueter
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
# env vars:
|
||||
# - UPDATE: update if available
|
||||
# - FORCE: forcefully update
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
. tools/cpm/common.sh
|
||||
|
||||
RETURN=0
|
||||
|
||||
filter() {
|
||||
TAGS=$(echo "$TAGS" | jq "[.[] | select(.name | test(\"$1\"; \"i\") | not)]") # vulkan
|
||||
TAGS=$(echo "$TAGS" | jq "[.[] | select(.name | test(\"$1\"; \"i\") | not)]")
|
||||
}
|
||||
|
||||
for PACKAGE in "$@"
|
||||
do
|
||||
usage() {
|
||||
cat << EOF
|
||||
Usage: $0 [uf] [PACKAGE]...
|
||||
Check a specific package or packages for updates.
|
||||
|
||||
Options:
|
||||
-u, --update Update the package if a new version is available.
|
||||
This will also update the hash if provided.
|
||||
|
||||
-f, --force Forcefully update the package version (implies -u)
|
||||
This is seldom useful, and should only be used in cases of
|
||||
severe corruption.
|
||||
|
||||
This project has defined the following as valid cpmfiles:
|
||||
EOF
|
||||
|
||||
for file in $CPMFILES; do
|
||||
echo "- $file"
|
||||
done
|
||||
|
||||
exit $RETURN
|
||||
}
|
||||
|
||||
while true; do
|
||||
case "$1" in
|
||||
(-uf|--force) UPDATE=true; FORCE=true; continue ;;
|
||||
(-u|--update) UPDATE=true; continue ;;
|
||||
(-h) usage ;;
|
||||
("$0") break ;;
|
||||
("") break ;;
|
||||
esac
|
||||
|
||||
PACKAGE="$1"
|
||||
|
||||
shift
|
||||
|
||||
export PACKAGE
|
||||
# shellcheck disable=SC1091
|
||||
. tools/cpm/package.sh
|
||||
|
|
@ -67,7 +97,6 @@ do
|
|||
|
||||
echo "-- * Version $LATEST available, current is $TAG"
|
||||
|
||||
export USE_TAG=true
|
||||
HASH=$(tools/cpm/hash.sh "$REPO" "$LATEST")
|
||||
|
||||
echo "-- * New hash: $HASH"
|
||||
|
|
|
|||
|
|
@ -7,22 +7,21 @@
|
|||
# CHANGE THESE FOR YOUR PROJECT! #
|
||||
##################################
|
||||
|
||||
# Which directories to search
|
||||
DIRS=". src"
|
||||
|
||||
# How many levels to go (3 is 2 subdirs max)
|
||||
MAXDEPTH=3
|
||||
|
||||
# shellcheck disable=SC2038
|
||||
# shellcheck disable=SC2016
|
||||
# shellcheck disable=SC2086
|
||||
[ -z "$PACKAGES" ] && PACKAGES=$(find $DIRS -maxdepth "$MAXDEPTH" -name cpmfile.json | xargs jq -s 'reduce .[] as $item ({}; . * $item)')
|
||||
# For your project you'll want to change this to define what dirs you have cpmfiles in
|
||||
# Remember to account for the MAXDEPTH variable!
|
||||
# Adding ./ before each will help to remove duplicates
|
||||
[ -z "$CPMFILES" ] && CPMFILES=$(find . ./src -maxdepth "$MAXDEPTH" -name cpmfile.json | sort | uniq)
|
||||
|
||||
# shellcheck disable=SC2016
|
||||
[ -z "$PACKAGES" ] && PACKAGES=$(echo "$CPMFILES" | xargs jq -s 'reduce .[] as $item ({}; . * $item)')
|
||||
|
||||
# For your project you'll want to change the PACKAGES call to include whatever locations you may use (externals, src, etc.)
|
||||
# Always include .
|
||||
LIBS=$(echo "$PACKAGES" | jq -j 'keys_unsorted | join(" ")')
|
||||
|
||||
export PACKAGES
|
||||
export CPMFILES
|
||||
export LIBS
|
||||
export DIRS
|
||||
export MAXDEPTH
|
||||
|
|
|
|||
|
|
@ -3,9 +3,6 @@
|
|||
# SPDX-FileCopyrightText: 2025 crueter
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
# env vars:
|
||||
# - UPDATE: fix hashes if needed
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
. tools/cpm/common.sh
|
||||
|
||||
|
|
|
|||
|
|
@ -7,4 +7,4 @@
|
|||
. tools/cpm/common.sh
|
||||
|
||||
# shellcheck disable=SC2086
|
||||
tools/cpm/fetch.sh $LIBS
|
||||
tools/cpm/fetch.sh "$@" $LIBS
|
||||
|
|
@ -18,9 +18,33 @@ ROOTDIR="$PWD"
|
|||
|
||||
TMP=$(mktemp -d)
|
||||
|
||||
# shellcheck disable=SC2034
|
||||
for PACKAGE in "$@"
|
||||
do
|
||||
usage() {
|
||||
cat << EOF
|
||||
Usage: $0 [PACKAGE]...
|
||||
Fetch the specified package or packages from their defined download locations.
|
||||
If the package is already cached, it will not be re-fetched.
|
||||
|
||||
This project has defined the following as valid cpmfiles:
|
||||
EOF
|
||||
|
||||
for file in $CPMFILES; do
|
||||
echo "- $file"
|
||||
done
|
||||
|
||||
exit 0
|
||||
}
|
||||
|
||||
while true; do
|
||||
case "$1" in
|
||||
(-h) usage ;;
|
||||
("$0") break ;;
|
||||
("") break ;;
|
||||
esac
|
||||
|
||||
PACKAGE="$1"
|
||||
|
||||
shift
|
||||
|
||||
export PACKAGE
|
||||
# shellcheck disable=SC1091
|
||||
. tools/cpm/package.sh
|
||||
|
|
|
|||
|
|
@ -6,10 +6,7 @@
|
|||
# shellcheck disable=SC1091
|
||||
. tools/cpm/common.sh
|
||||
|
||||
# shellcheck disable=SC2086
|
||||
FILES=$(find $DIRS -maxdepth "$MAXDEPTH" -name cpmfile.json)
|
||||
|
||||
for file in $FILES; do
|
||||
for file in $CPMFILES; do
|
||||
jq --indent 4 < "$file" > "$file".new
|
||||
mv "$file".new "$file"
|
||||
done
|
||||
|
|
|
|||
|
|
@ -6,20 +6,94 @@
|
|||
# usage: hash.sh repo tag-or-sha
|
||||
# env vars: GIT_HOST, USE_TAG (use tag instead of sha), ARTIFACT (download artifact with that name instead of src archive)
|
||||
|
||||
REPO="$1"
|
||||
[ -z "$GIT_HOST" ] && GIT_HOST=github.com
|
||||
RETURN=0
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
Usage: $0 [-a|--artifact ARTIFACT] [-g|--host GIT_HOST] [REPO] [REF]
|
||||
Get the hash of a package.
|
||||
|
||||
REPO must be in the form of OWNER/REPO, and REF must be a commit sha, branch, or tag.
|
||||
|
||||
Options:
|
||||
-g, --host <GIT_HOST> What Git host to use (defaults to github.com)
|
||||
|
||||
-a, --artifact <ARTIFACT> The artifact to download (implies -t)
|
||||
If -t is specified but not -a, fetches a tag archive.
|
||||
If ARTIFACT is specified but is null,
|
||||
|
||||
EOF
|
||||
exit "$RETURN"
|
||||
}
|
||||
|
||||
die() {
|
||||
echo "$@" >&2
|
||||
RETURN=1 usage
|
||||
}
|
||||
|
||||
artifact() {
|
||||
if [ $# -lt 2 ]; then
|
||||
die "You must specify a valid artifact."
|
||||
fi
|
||||
|
||||
shift
|
||||
|
||||
ARTIFACT="$1"
|
||||
}
|
||||
|
||||
host() {
|
||||
if [ $# -lt 2 ]; then
|
||||
die "You must specify a valid Git host."
|
||||
fi
|
||||
|
||||
shift
|
||||
|
||||
GIT_HOST="$1"
|
||||
}
|
||||
|
||||
# this is a semi-hacky way to handle long/shortforms
|
||||
while true; do
|
||||
case "$1" in
|
||||
-[a-z]*)
|
||||
opt=$(echo "$1" | sed 's/^-//')
|
||||
while [ -n "$opt" ]; do
|
||||
# cut out first char from the optstring
|
||||
char=$(echo "$opt" | cut -c1)
|
||||
opt=$(echo "$opt" | cut -c2-)
|
||||
|
||||
case "$char" in
|
||||
a) artifact "$@" ;;
|
||||
g) host "$@" ;;
|
||||
h) usage ;;
|
||||
*) die "Invalid option -$char" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
;;
|
||||
--artifact) artifact "$@" ;;
|
||||
--host) host "$@" ;;
|
||||
--help) usage ;;
|
||||
--*) die "Invalid option $1" ;;
|
||||
"$0" | "") break ;;
|
||||
*)
|
||||
{ [ -z "$REPO" ] && REPO="$1"; } || REF="$1"
|
||||
;;
|
||||
esac
|
||||
|
||||
shift
|
||||
done
|
||||
|
||||
[ -z "$REPO" ] && die "A valid repository must be provided."
|
||||
[ -z "$REF" ] && die "A valid reference must be provided."
|
||||
|
||||
GIT_HOST=${GIT_HOST:-github.com}
|
||||
GIT_URL="https://$GIT_HOST/$REPO"
|
||||
|
||||
if [ "$USE_TAG" = "true" ]; then
|
||||
if [ -z "$ARTIFACT" ] || [ "$ARTIFACT" = "null" ]; then
|
||||
URL="${GIT_URL}/archive/refs/tags/$2.tar.gz"
|
||||
else
|
||||
URL="${GIT_URL}/releases/download/$2/${ARTIFACT}"
|
||||
fi
|
||||
if [ -z "$ARTIFACT" ] || [ "$ARTIFACT" = "null" ]; then
|
||||
URL="${GIT_URL}/archive/$REF.tar.gz"
|
||||
else
|
||||
URL="${GIT_URL}/archive/$2.zip"
|
||||
URL="${GIT_URL}/releases/download/$REF/$ARTIFACT"
|
||||
fi
|
||||
|
||||
SUM=$(wget -q "$URL" -O - | sha512sum)
|
||||
|
||||
echo "$SUM" | cut -d " " -f1
|
||||
|
|
|
|||
|
|
@ -3,9 +3,6 @@
|
|||
# SPDX-FileCopyrightText: 2025 crueter
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
# env vars:
|
||||
# - UPDATE: fix hashes if needed
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
. tools/cpm/common.sh
|
||||
|
||||
|
|
@ -14,7 +11,7 @@
|
|||
# shellcheck disable=SC2153
|
||||
JSON=$(echo "$PACKAGES" | jq -r ".\"$PACKAGE\" | select( . != null )")
|
||||
|
||||
[ -z "$JSON" ] && echo "!! No cpmfile definition for $PACKAGE" && exit 1
|
||||
[ -z "$JSON" ] && echo "!! No cpmfile definition for $PACKAGE" >&2 && exit 1
|
||||
|
||||
# unset stuff
|
||||
export PACKAGE_NAME="null"
|
||||
|
|
@ -134,13 +131,13 @@ elif [ "$REPO" != "null" ]; then
|
|||
DOWNLOAD="${GIT_URL}/archive/refs/tags/${TAG}.tar.gz"
|
||||
fi
|
||||
elif [ "$SHA" != "null" ]; then
|
||||
DOWNLOAD="${GIT_URL}/archive/${SHA}.zip"
|
||||
DOWNLOAD="${GIT_URL}/archive/${SHA}.tar.gz"
|
||||
else
|
||||
if [ "$BRANCH" = null ]; then
|
||||
BRANCH=master
|
||||
fi
|
||||
|
||||
DOWNLOAD="${GIT_URL}/archive/refs/heads/${BRANCH}.zip"
|
||||
DOWNLOAD="${GIT_URL}/archive/refs/heads/${BRANCH}.tar.gz"
|
||||
fi
|
||||
else
|
||||
echo "!! No repo or URL defined for $PACKAGE"
|
||||
|
|
|
|||
|
|
@ -8,8 +8,8 @@
|
|||
. tools/cpm/common.sh
|
||||
|
||||
# shellcheck disable=SC2086
|
||||
JSON=$(find $DIRS -maxdepth "$MAXDEPTH" -name cpmfile.json -exec grep -l "$1" {} \;)
|
||||
JSON=$(echo "$CPMFILES" | xargs grep -l "$1")
|
||||
|
||||
[ -z "$JSON" ] && echo "!! No cpmfile definition for $1"
|
||||
[ -z "$JSON" ] && echo "!! No cpmfile definition for $1" >&2 && exit 1
|
||||
|
||||
echo "$JSON"
|
||||
|
|
@ -1,11 +1,7 @@
|
|||
#!/bin/sh -e
|
||||
|
||||
# SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
|
||||
# SPDX-FileCopyrightText: Copyright 2025 crueter
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
# fd is slightly faster on NVMe (the syntax sux though)
|
||||
if command -v fd > /dev/null; then
|
||||
fd . tools -esh -x shellcheck
|
||||
else
|
||||
find tools -name "*.sh" -exec shellcheck -s sh {} \;
|
||||
fi
|
||||
# do NOT use fd in scripts, PLEASE
|
||||
find tools -name "*.sh" -exec shellcheck -s sh {} \;
|
||||
|
|
|
|||
|
|
@ -1,6 +1,16 @@
|
|||
#!/bin/sh -e
|
||||
|
||||
# SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
|
||||
# SPDX-FileCopyrightText: Copyright 2025 crueter
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
wget -O CMakeModules/CPM.cmake https://github.com/cpm-cmake/CPM.cmake/releases/latest/download/CPM.cmake
|
||||
# updates CPMUtil, its docs, and related tools from the latest release
|
||||
|
||||
if command -v zstd > /dev/null; then
|
||||
EXT=tar.zst
|
||||
else
|
||||
EXT=tar.gz
|
||||
fi
|
||||
|
||||
wget "https://git.crueter.xyz/CMake/CPMUtil/releases/download/continuous/CPMUtil.$EXT"
|
||||
tar xf "CPMUtil.$EXT"
|
||||
rm "CPMUtil.$EXT"
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue