Conversation
There was a problem hiding this comment.
Code Review
This pull request updates numerous package.json files to exclude .proto files from published npm packages and introduces a command to delete the build/protos/google directory during compilation. The review feedback correctly identifies a critical issue: explicitly deleting the google directory removes essential JSON descriptors required by the libraries at runtime, which would break the generated clients. Since the .proto files are already excluded via the files array in package.json, the reviewer recommends removing the destructive rm -rf commands from the compile and prepare scripts to ensure library stability.
| "lint": "gts check src samples test", | ||
| "clean": "gts clean", | ||
| "compile": "tsc -p . && cp src/*.json build/src && cp -r test/fixtures build/test && cp -r protos build/", | ||
| "compile": "tsc -p . && cp src/*.json build/src && cp -r test/fixtures build/test && cp -r protos build/ && rm -rf build/protos/google", |
There was a problem hiding this comment.
The command rm -rf build/protos/google is too broad. It deletes the entire google directory tree within build/protos, which typically contains not only .proto files but also the .json descriptors required by the library at runtime (e.g., google/api/http.json, google/longrunning/operations.json). Deleting these will break the library's ability to load these descriptors. Since you have already added !build/protos/**/*.proto to the files array in package.json, the .proto files will be excluded from the published npm package automatically. If the intention of deleting them during compile is to ensure tests don't rely on them, you should specifically target .proto files while preserving .json files.
| "compile": "tsc -p . && cp src/*.json build/src && cp -r test/fixtures build/test && cp -r protos build/ && rm -rf build/protos/google", | |
| "compile": "tsc -p . && cp src/*.json build/src && cp -r test/fixtures build/test && cp -r protos build/", |
| "compile-http-protos": "pbjs -t static-module -r http_proto --keep-case google/api/http.proto -p ./protos > protos/http.js && pbts protos/http.js -o protos/http.d.ts", | ||
| "fix": "gts fix", | ||
| "prepare": "npm run compile && cd ../tools && npm i && npm run compile && cd ../gax && pwd && node ../tools/build/src/prepublish.js . && mkdirp build/protos && cp -r protos/* build/protos/ && npm run minify-proto-json", | ||
| "prepare": "npm run compile && cd ../tools && npm i && npm run compile && cd ../gax && pwd && node ../tools/build/src/prepublish.js . && mkdirp build/protos && cp -r protos/* build/protos/ && rm -rf build/protos/google && npm run minify-proto-json", |
There was a problem hiding this comment.
As noted in the compile script, rm -rf build/protos/google will delete required JSON descriptors. Additionally, this prepare script redundantly copies protos to build/protos after npm run compile has already done so. It is safer to rely on the files exclusion in package.json to prevent publishing .proto files while preserving the necessary JSON files.
| "prepare": "npm run compile && cd ../tools && npm i && npm run compile && cd ../gax && pwd && node ../tools/build/src/prepublish.js . && mkdirp build/protos && cp -r protos/* build/protos/ && rm -rf build/protos/google && npm run minify-proto-json", | |
| "prepare": "npm run compile && cd ../tools && npm i && npm run compile && cd ../gax && pwd && node ../tools/build/src/prepublish.js . && mkdirp build/protos && cp -r protos/* build/protos/ && npm run minify-proto-json", |
| @@ -25,7 +26,7 @@ | |||
| ], | |||
| "scripts": { | |||
| "clean": "gts clean", | |||
There was a problem hiding this comment.
Deleting build/protos/google will remove the package's own proto descriptors (e.g., google/cloud/<service>/v1/<service>.json) if they are located within that directory structure. This will break the generated client at runtime. The exclusion of .proto files from the npm package is already correctly handled by the files array entry !build/protos/**/*.proto. You should remove the rm -rf command to ensure the necessary JSON descriptors are preserved in the build.
| "clean": "gts clean", | |
| "compile": "tsc -p . && cp -r protos build/{% if not api.legacyProtoLoad %} && minifyProtoJson{% endif %}", |
There was a problem hiding this comment.
I notice this change is inside core. I'm somewhat concerned about that.
| "babel": "babel esm --out-dir build/cjs --ignore \"esm/**/*.d.ts\" --extensions \".ts\" --out-file-extension .cjs --copy-files", | ||
| "compile:cjs": "tsc -p ./tsconfig.json && npm run babel", | ||
| "compile": "npm run compile:esm && rm -rf esm/src/json-helper.d.cts && npm run compile:cjs && rm -rf build/protos && cp -r protos build/protos", | ||
| "compile": "npm run compile:esm && rm -rf esm/src/json-helper.d.cts && npm run compile:cjs && rm -rf build/protos && cp -r protos build/protos && rm -rf build/protos/google", |
There was a problem hiding this comment.
As with the CJS template, rm -rf build/protos/google is likely to delete required JSON descriptors, breaking the library. The files array already excludes .proto files from the package. Recommend removing this deletion step from the compile script.
| "compile": "npm run compile:esm && rm -rf esm/src/json-helper.d.cts && npm run compile:cjs && rm -rf build/protos && cp -r protos build/protos && rm -rf build/protos/google", | |
| "compile": "npm run compile:esm && rm -rf esm/src/json-helper.d.cts && npm run compile:cjs && rm -rf build/protos && cp -r protos build/protos", |
| ], | ||
| "scripts": { | ||
| "compile": "tsc -p . && cp -r proto* build/ && cp -r testproxy/proto* build/testproxy/", | ||
| "compile": "tsc -p . && cp -r proto* build/ && rm -rf build/protos/google && cp -r testproxy/proto* build/testproxy/", |
There was a problem hiding this comment.
Deleting build/protos/google will remove the Bigtable proto descriptors, breaking the library. The files array already handles the exclusion of .proto files from the published package.
| "compile": "tsc -p . && cp -r proto* build/ && rm -rf build/protos/google && cp -r testproxy/proto* build/testproxy/", | |
| "compile": "tsc -p . && cp -r proto* build/ && cp -r testproxy/proto* build/testproxy/", |
danieljbruce
left a comment
There was a problem hiding this comment.
We should address this item
| @@ -25,7 +26,7 @@ | |||
| ], | |||
| "scripts": { | |||
| "clean": "gts clean", | |||
There was a problem hiding this comment.
I notice this change is inside core. I'm somewhat concerned about that.
danieljbruce
left a comment
There was a problem hiding this comment.
We need to make sure prompting does not change the string that is fed into protos.
| "scripts": { | ||
| "clean": "gts clean", | ||
| "compile": "tsc -p . && cp -r protos build/{% if not api.legacyProtoLoad %} && minifyProtoJson{% endif %}", | ||
| "compile": "tsc -p . && cp -r protos build/ && rm -rf build/protos/google{% if not api.legacyProtoLoad %} && minifyProtoJson{% endif %}", |
There was a problem hiding this comment.
This generated code here is concerning. The string for the protos command should not change.
Description
This pull request addresses issue googleapis/librarian#4820. The
.protofiles should be the only files removed from the build so that customers enjoy reduced package dependency size.Note that we also change
npm run compileso thatbuild/protosdoesn't get created. This is advantageous because it ensures that when testing we will discover that the build fails earlier when the tests don't work instead of the build failing when we try to publish the package.Impact
Will speed up npm install for customers and reduce customer burden.
Testing
It is worth calling out that it is kind of difficult to test package publishing. We could cut a pre-release and see what happens, but maybe this isn't necessary since npm will stop publishing if any changes break the build anyway. We also considered the impact of customers reaching into the library to reference the .proto files, but these files are unlikely to be useful to customers anyway.