ci : apply ccache-clear with older/min/dry-run to all ccache jobs (#27602)
* ci : apply ccache-clear with older/min/dry-run to all ccache jobs Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731 * ci : install gh in ccache-clear if missing (container jobs) The ccache-clear action relies on the gh CLI, which is not present in container-based jobs. Install it on demand so those jobs can clear caches. Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731 * ci : install gh via apt repo in ccache-clear The install.sh script used previously is no longer served (404). Switch to the official GitHub CLI apt repository, which is still available. Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731 * ci : pass --repo to gh cache commands in ccache-clear In container jobs gh cannot auto-detect the repository from git, so gh cache list/delete fail with 'failed to run git: not a git repository'. Pass the repository explicitly via --repo using GITHUB_REPOSITORY. Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731 * ci : drop -new suffix from vulkan ccache key The -new suffix was only needed to force a fresh cache. With ccache-clear now evicting stale caches, the original key can be used again. The old ccache-vulkan-ubuntu-24.04-arm-new entries still match the ccache-clear key prefix and are cleaned up automatically. Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731 * ci : fix ccache-clear date parsing on macOS (BSD date) macOS ships BSD date, which has no -d option. The older cutoff check was silently disabled there: 'date: illegal option -- d' errors in the log and the loop was only stopped by the min limit, risking deletion of caches not older than the cutoff (e.g. saved by a concurrent job). Parse the ISO-8601 timestamps with GNU date when available and fall back to BSD date otherwise (TZ=UTC, fractional seconds dropped). Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731 * ci : extract ccache-clear logic into scripts/ccache-clear.sh The composite action now consists of a dedicated step that installs the GitHub CLI when missing (e.g. in container jobs) and a thin step that calls the new script. The script follows the make-release-checks.sh conventions (usage/env header, set -euo pipefail, CLI flags) and only checks that gh is available. The action inputs are unchanged, so the workflow steps are untouched. Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731 * ci : remove unused apple ccaches
This commit is contained in:
Georgi Gerganov
@@ -21,68 +21,30 @@ inputs:
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Install GitHub CLI if missing
|
||||
shell: bash
|
||||
run: |
|
||||
# e.g. in container jobs, where it is not preinstalled
|
||||
if ! command -v gh >/dev/null 2>&1; then
|
||||
echo "GitHub CLI not found, installing..."
|
||||
if ! command -v curl >/dev/null 2>&1; then
|
||||
apt-get update >/dev/null 2>&1 || true
|
||||
apt-get install -y curl >/dev/null 2>&1 || true
|
||||
fi
|
||||
mkdir -p -m 755 /etc/apt/keyrings
|
||||
curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | tee /etc/apt/keyrings/githubcli-archive-keyring.gpg >/dev/null
|
||||
chmod go+r /etc/apt/keyrings/githubcli-archive-keyring.gpg
|
||||
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" > /etc/apt/sources.list.d/github-cli.list
|
||||
apt-get update >/dev/null 2>&1 || true
|
||||
apt-get install -y gh || { echo "Failed to install GitHub CLI (gh)" >&2; exit 1; }
|
||||
fi
|
||||
command -v gh >/dev/null 2>&1 || { echo "GitHub CLI (gh) is required but could not be installed" >&2; exit 1; }
|
||||
|
||||
- name: Clear caches
|
||||
shell: bash
|
||||
env:
|
||||
CLEAR_KEY: ${{ inputs.key }}
|
||||
CLEAR_OLDER: ${{ inputs.older }}
|
||||
CLEAR_MIN: ${{ inputs.min }}
|
||||
CLEAR_DRY_RUN: ${{ inputs.dry-run }}
|
||||
run: |
|
||||
# Convert a duration (e.g. 90m, 1h, 1d, plain seconds) to seconds
|
||||
to_seconds() {
|
||||
local val="$1"
|
||||
[[ "$val" =~ ^[0-9]+$ ]] && { echo "$val"; return 0; }
|
||||
local num="${val%?}" unit="${val: -1}" mult
|
||||
[[ "$num" =~ ^[0-9]+$ ]] || return 1
|
||||
case "$unit" in
|
||||
s) mult=1 ;;
|
||||
m) mult=60 ;;
|
||||
h) mult=3600 ;;
|
||||
d) mult=86400 ;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
echo $((num * mult))
|
||||
}
|
||||
|
||||
[[ "$CLEAR_MIN" =~ ^[0-9]+$ ]] || { echo "Invalid min value: $CLEAR_MIN" >&2; exit 1; }
|
||||
[[ "$CLEAR_DRY_RUN" =~ ^(true|false)$ ]] || { echo "Invalid dry-run value: $CLEAR_DRY_RUN" >&2; exit 1; }
|
||||
|
||||
CACHES=$(gh cache list --key "ccache-$CLEAR_KEY" --json id,key,createdAt --jq '.[] | [.createdAt, .id, .key] | @tsv' 2>/dev/null | LC_ALL=C sort)
|
||||
if [ -z "$CACHES" ]; then
|
||||
echo "No caches found with key prefix: $CLEAR_KEY"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
TOTAL=$(( $(wc -l <<< "$CACHES") ))
|
||||
|
||||
echo "Found $TOTAL cache(s) with key prefix: $CLEAR_KEY (oldest first):"
|
||||
while IFS=$'\t' read -r CREATED ID KEY; do
|
||||
printf ' %s %s %s\n' "$CREATED" "$ID" "$KEY"
|
||||
done <<< "$CACHES"
|
||||
|
||||
CUTOFF=""
|
||||
if [ -n "$CLEAR_OLDER" ]; then
|
||||
OLDER_SECONDS=$(to_seconds "$CLEAR_OLDER") || { echo "Invalid older value: $CLEAR_OLDER (expected e.g. 90m, 1h, 1d)" >&2; exit 1; }
|
||||
CUTOFF=$(( $(date +%s) - OLDER_SECONDS ))
|
||||
fi
|
||||
|
||||
# Caches are sorted oldest first
|
||||
DELETED=0
|
||||
while IFS=$'\t' read -r CREATED ID KEY; do
|
||||
if [ -n "$CUTOFF" ] && [ "$(date -d "$CREATED" +%s)" -ge "$CUTOFF" ]; then
|
||||
echo "Rest are not older than $CLEAR_OLDER, stopping"
|
||||
break
|
||||
fi
|
||||
if [ $((TOTAL - DELETED - 1)) -lt "$CLEAR_MIN" ]; then
|
||||
echo "Keeping at least $CLEAR_MIN cache(s), stopping"
|
||||
break
|
||||
fi
|
||||
if [ "$CLEAR_DRY_RUN" = "true" ]; then
|
||||
echo "Would delete cache: $ID ($KEY)"
|
||||
else
|
||||
echo "Deleting cache: $ID ($KEY)"
|
||||
gh cache delete "$ID"
|
||||
fi
|
||||
DELETED=$((DELETED + 1))
|
||||
done <<< "$CACHES"
|
||||
bash scripts/ccache-clear.sh \
|
||||
--key "${{ inputs.key }}" \
|
||||
--older "${{ inputs.older }}" \
|
||||
--min "${{ inputs.min }}" \
|
||||
${{ inputs.dry-run == 'true' && '--dry-run' || '' }}
|
||||
|
||||
Reference in New Issue
Block a user