ci: add compile-only PR build job for spark-4.1

Adds a new build-spark-4.1 job to pr_build_linux.yml that runs ./mvnw install -DskipTests -Dmaven.test.skip=true -Pspark-4.1. Verifies that the new profile and shim trees keep compiling on every PR without enabling any spark-4.1 tests in CI.

Avoids the existing lint-java matrix because that job runs -Psemanticdb scalafix and semanticdb-scalac_2.13.17 is not yet published. Avoids the linux-test matrix because Comet's existing test suites have known failures under the spark-4.1 profile that are deferred to follow-up PRs.

The job is independent: needs only the lint gate, doesn't share artifacts with build-native, and doesn't change any other job.

Author: andygrove

.github/workflows/pr_build_linux.yml

@@ -140,6 +140,38 @@ jobs:
         run: |
           ./dev/ci/check-working-tree-clean.sh
 
+  # Compile-only verification for Spark 4.1. Tests are intentionally skipped: the spark-4.1
+  # profile is currently a build target only, and several runtime/test failures are tracked
+  # in follow-up PRs. Excluded from lint-java because semanticdb-scalac_2.13.17 is not yet
+  # published and the lint job activates -Psemanticdb.
+  build-spark-4.1:
+    needs: lint
+    name: Build Spark 4.1, JDK 17
+    runs-on: ubuntu-latest
+    container:
+      image: amd64/rust
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Setup Rust & Java toolchain
+        uses: ./.github/actions/setup-builder
+        with:
+          rust-version: ${{ env.RUST_VERSION }}
+          jdk-version: 17
+
+      - name: Cache Maven dependencies
+        uses: actions/cache@v5
+        with:
+          path: |
+            ~/.m2/repository
+            /root/.m2/repository
+          key: ${{ runner.os }}-java-maven-${{ hashFiles('**/pom.xml') }}-spark-4.1-build
+          restore-keys: |
+            ${{ runner.os }}-java-maven-
+
+      - name: Compile (skip tests)
+        run: ./mvnw -B install -DskipTests -Dmaven.test.skip=true -Pspark-4.1
+
   # Build native library once and share with all test jobs
   build-native:
     needs: lint