Skip to content

Add S3 checksum properties to director blobstore configuration#2726

Merged
ragaskar merged 1 commit intomainfrom
topic/nmra/add-s3-checksum-properties
May 5, 2026
Merged

Add S3 checksum properties to director blobstore configuration#2726
ragaskar merged 1 commit intomainfrom
topic/nmra/add-s3-checksum-properties

Conversation

@Alphasite
Copy link
Copy Markdown
Contributor

Summary

  • Add request_checksum_calculation_enabled, response_checksum_calculation_enabled, and uploader_request_checksum_calculation_enabled to the director manifest spec for S3 blobstores.
  • Wire these properties through the director.yml.erb template.
  • Pass these properties to the s3cli config file via the S3cliBlobstoreClient.

Test plan

  • Unit tests added to s3cli_blobstore_client_spec.rb and director.yml.erb_spec.rb.

Made with Cursor

Copilot AI review requested due to automatic review settings May 5, 2026 18:56
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 5, 2026

Walkthrough

Three new S3 blobstore checksum configuration properties were added: blobstore.request_checksum_calculation_enabled, blobstore.response_checksum_calculation_enabled, and blobstore.uploader_request_checksum_calculation_enabled. They were added to the job spec, conditionally rendered into blobstore.options in the director.yml.erb template for the S3 provider, propagated into S3cliBlobstoreClient's generated s3cli config, and covered by updated unit/template specs asserting their presence and values.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The PR description is missing several required sections from the template: contextual information/links, comprehensive test list, release notes description, breaking change assessment, and team tagging. Add the missing template sections: contextual links, detailed test results, release notes summary, breaking change confirmation, and team/pair tags.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately and concisely describes the main change: adding S3 checksum properties to director blobstore configuration.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch topic/nmra/add-s3-checksum-properties

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@Alphasite Alphasite marked this pull request as draft May 5, 2026 18:56
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the director’s S3 blobstore configuration so checksum-calculation settings from the manifest are carried through to the rendered director.yml and into the s3cli config consumed by the director blobstore client.

Changes:

  • Added three optional S3 checksum properties to the director job spec.
  • Wired those properties into jobs/director/templates/director.yml.erb for S3 blobstores.
  • Added unit coverage for both template rendering and S3cliBlobstoreClient config serialization.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/bosh-director/spec/unit/bosh/director/blobstore/s3cli_blobstore_client_spec.rb Adds coverage that checksum flags are written into the generated s3cli config.
src/bosh-director/lib/bosh/director/blobstore/s3cli_blobstore_client.rb Passes the new checksum options through to the s3cli JSON config.
spec/director.yml.erb_spec.rb Verifies the rendered director config includes the new S3 checksum properties.
jobs/director/templates/director.yml.erb Maps new manifest properties into S3 blobstore options in rendered director config.
jobs/director/spec Declares the new optional manifest properties for the director job.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Currently the s3 blobstore cli has compatbaility issues with dell ecs in
certain configurations. There are flags to disable it but there need to
be wired into the director manfiest as it maintains an allow list of properties
which rejects unknown properties.

Co-authored-by: Cursor <cursoragent@cursor.com>
Copy link
Copy Markdown
Contributor

@ragaskar ragaskar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think ideally we rewrite the commit to be a little more descriptive about the problem we are solving with this change, but otherwise lgtm

@Alphasite Alphasite force-pushed the topic/nmra/add-s3-checksum-properties branch from 01651e2 to 9689a2b Compare May 5, 2026 20:34
coderabbitai[bot]
coderabbitai Bot previously requested changes May 5, 2026
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@spec/director.yml.erb_spec.rb`:
- Around line 144-158: Add a complementary spec context that sets
merged_manifest_properties['blobstore']['request_checksum_calculation_enabled'],
['response_checksum_calculation_enabled'], and
['uploader_request_checksum_calculation_enabled'] to true and asserts
parsed_yaml['blobstore']['options'][...] equals true for each key; this
complements the existing false-case and ensures the ERB template handles true
values (look for the existing context block using merged_manifest_properties and
the expectations on parsed_yaml['blobstore']['options'] to copy structure).

In
`@src/bosh-director/spec/unit/bosh/director/blobstore/s3cli_blobstore_client_spec.rb`:
- Around line 136-151: Add a complementary spec that sets
request_checksum_calculation_enabled, response_checksum_calculation_enabled and
uploader_request_checksum_calculation_enabled to true and asserts those keys are
true in the parsed stored_config_file; specifically, duplicate the existing
context block but call described_class.new(options.merge({
request_checksum_calculation_enabled: true,
response_checksum_calculation_enabled: true,
uploader_request_checksum_calculation_enabled: true })) then parse JSON from
stored_config_file[0] and expect config['request_checksum_calculation_enabled'],
config['response_checksum_calculation_enabled'] and
config['uploader_request_checksum_calculation_enabled'] to eq(true).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 6a5f1881-bef0-441f-baf8-c18d46421b53

📥 Commits

Reviewing files that changed from the base of the PR and between 01651e2 and 9689a2b.

📒 Files selected for processing (5)
  • jobs/director/spec
  • jobs/director/templates/director.yml.erb
  • spec/director.yml.erb_spec.rb
  • src/bosh-director/lib/bosh/director/blobstore/s3cli_blobstore_client.rb
  • src/bosh-director/spec/unit/bosh/director/blobstore/s3cli_blobstore_client_spec.rb

Comment thread spec/director.yml.erb_spec.rb
@github-project-automation github-project-automation Bot moved this from Pending Merge | Prioritized to Waiting for Changes | Open for Contribution in Foundational Infrastructure Working Group May 5, 2026
@github-project-automation github-project-automation Bot moved this from Waiting for Changes | Open for Contribution to Pending Merge | Prioritized in Foundational Infrastructure Working Group May 5, 2026
@ragaskar ragaskar dismissed coderabbitai[bot]’s stale review May 5, 2026 20:56

We don't think there's value in these suggestions (basically adding a non-interesting case for complete-ness sake).

@ragaskar ragaskar merged commit b9a3917 into main May 5, 2026
22 checks passed
@github-project-automation github-project-automation Bot moved this from Pending Merge | Prioritized to Done in Foundational Infrastructure Working Group May 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

3 participants