"The package.json file is too large. Size limit is 384000 bytes" when trying to push npm package to azure artifacts

Question

"The package.json file is too large. Size limit is 384000 bytes" when trying to push npm package to azure artifacts

Witold Kupś 10

Hello, we are migrating to azure artifacts from our local hosted nexus.
We have one npm lib, which has quite big package json, and also a couple of bigger files (>1MB).
Is it somehow possible to push that to azure, and then use as a normal npm package in our projects?
if not - why is there such a limitation? I could not find anywhere anything regarding package.json/single file limitation, the only I found was for the whole package (tgz has 8.4 MB).

Not to mention this https://learn.microsoft.com/en-us/azure/devops/artifacts/reference/limits?view=azure-devops#size-limits

Npm packages: limited to 500 MB per file.
whereas I have also error after cutting package.json that some file has size higher than 1MB

3 answers

Your answer

Answer 1

Erkan Sahin 840

Azure Artifacts has a size limit of 500 MB per npm package file. If your package exceeds this, consider the following:

Optimize Your Package
Reduce the size by removing unnecessary files or splitting it into smaller packages.

Use Universal Packages
For large binary files, Azure supports Universal Packages up to 4 TB.

Set Retention Policies
Automatically delete older package versions to manage storage.

In essence, review your npm package to fit Azure's size limits, and consider using Azure's features like Universal Packages and retention policies to manage large files and storage efficiently.

Witold Kupś 10 Reputation points

2023-08-23T08:37:01.64+00:00
Did you notice I mentioned the error is about the NPM package, which has 8.4 MB packed, unpacked it is 40 mb, and the error is regarding:

package.json, because it has 1mb, and the hidden limit is 340 KB,

a single file - I have a couple of files which are heavier than 1MB, and that is the limit.

Not to mention that I linked the same what you wrote.
Erkan Sahin 840 Reputation points

2023-08-23T08:57:01.47+00:00
I apologize for the oversight. Let's address the specific issues you mentioned:

package.json Size Limit:

The package.json file has a hidden size limit of 340 KB, and yours is 1 MB. This is indeed a limitation that isn't widely documented. To address this:

Optimize: Review the package.json file to see if there are any unnecessary dependencies, scripts, or metadata that can be removed.

Split: If you have large configuration objects or data embedded in package.json, consider moving them to separate configuration or data files.

Single File Size Limit:

You mentioned that there's a 1 MB limit on individual files, and you have several files exceeding this limit.

Optimize: For larger files, especially if they are source code, consider refactoring or splitting them into smaller modules.

Compression: If the files are assets (like images or binaries), consider using compression techniques or tools to reduce their size.

External Storage: For non-code assets, you might want to host them externally (e.g., on a CDN) and reference them in your project, rather than including them directly in the npm package.

Overall Package Size:

While the packed size is 8.4 MB (which is under the widely known limit), the unpacked size is 40 MB. It's essential to be aware of both packed and unpacked sizes, as different systems or processes might have different constraints.

Given these constraints, it's crucial to optimize both the structure and content of your npm package. The goal is to ensure that individual files and the overall package adhere to the size limits while retaining all necessary functionality.

Again, I apologize for the oversight in the initial response.
Witold Kupś 10 Reputation points

2023-08-23T08:59:57.9233333+00:00

What if I cannot optimize (really). I have a speciffic npm package, extracted from the application, and that structure is required. It is not like the I have trash in the package.json.
Erkan Sahin 840 Reputation points

2023-08-23T09:04:25.13+00:00
If you can't optimize the npm package and need to maintain its structure:

Set up your own private npm registry, like Verdaccio, to bypass Azure Artifacts' size limitations.

Consider using other package hosting services, such as GitHub Packages or JFrog Artifactory, which might offer more flexibility.

Host larger files externally, like on a CDN, and reference them in your npm package.

In essence, if optimization isn't an option, consider alternative hosting solutions or external references to meet the size constraints. I hope you will mark my answer :-)

Witold Kupś 10

Right now I have trimmed every file which had more than 1mb, the highest (checked with find has 660KB). I have another problem like below

npm ERR! 500 Internal Server Error - TF400898: An Internal Error Occurred. Activity Id: npm_***. (DevOps Activity ID: F19E5DC6-1790-4C28-B53A-E3AE16A1F82F) - PUT https://pkgs.dev.azure.com/xxxx/_packaging/TestFeed/npm/registry/@xxx%2fbundle - Storage Account: sr7pkgsprodsu3weus15, httpStatusCode: 400, ErrorCode: EntityTooLarge, ErrorMessage: 1:The entity is larger than the maximum allowed size (1MB).
npm ERR! RequestId:npm_***
npm ERR! Time:2023-08-23T09:02:09.9943469Z, ServiceRequestID: npm_***
npm ERR! Operation batch returned non-returnable status code: BadRequest
npm ERR! Element 1 in the batch returned an unexpected response code.

It looks to me that the limit for the package size is 1MB then, not 500...

another try, just pushing the raw package.json

$ npm publish
npm notice
npm notice 📦  @xxx/bundle@0.14.10
npm notice === Tarball Contents ===
npm notice 360.1kB package.json
npm notice === Tarball Details ===
npm notice name:          @xxx/bundle
npm notice version:       0.14.10
npm notice filename:      xxx-bundle-0.14.10.tgz
npm notice package size:  23.2 kB
npm notice unpacked size: 360.1 kB
npm notice shasum:        3400cdc05a295daae9a14522e2b5e05b0dd11a75
npm notice integrity:     sha512-qtIui4UvUrNdx[...]B9I8am9+Npnnw==
npm notice total files:   1
npm notice
npm notice Publishing to https://pkgs.dev.azure.com/xxxx/_packaging/TestFeed/npm/registry/ with tag latest and default access
npm ERR! code E500
npm ERR! 500 Internal Server Error - TF400898: An Internal Error Occurred. Activity Id: npm_***. (DevOps Activity ID: D42ABAC4-8519-4BE5-A90F-F4150E4726F6) - PUT https://pkgs.dev.azure.com/xxxx/_packaging/TestFeed/npm/registry/@xxx%2fbundle - Storage Account: sr7pkgsprodsu3weus15, httpStatusCode: 400, ErrorCode: EntityTooLarge, ErrorMessage: 1:The entity is larger than the maximum allowed size (1MB).
npm ERR! RequestId:npm_***
npm ERR! Time:2023-08-23T09:16:39.0345759Z, ServiceRequestID: npm_***
npm ERR! Operation batch returned non-returnable status code: BadRequest
npm ERR! Element 1 in the batch returned an unexpected response code.

Witold Kupś 10 Reputation points

2023-08-23T09:17:37.1933333+00:00

I will @Erkan Sahin , but I really would like to have the solution for the issue, and the answer that it is not possible because of some undocumented strange unrealistic limitation does not looks valuable. And btw, we want to migrate from the private solution (we migrate from our own nexus registry)
Erkan Sahin 840 Reputation points

2023-08-23T09:30:53.5666667+00:00

The error indicates the overall package or a specific entity in the request exceeds the allowed size. Check the overall packed size of the npm package, ensuring it’s within Azure Artifacts’ limit. If pushing multiple packages or updates, try them individually. Review Azure DevOps for any size limits set at the project or organization level. Since you’re migrating from Nexus, consider Azure DevOps’ import tools for smoother migration.

Best Regards
Witold Kupś 10 Reputation points

2023-08-23T09:48:04.8433333+00:00

Erkan, once again.
npm notice package size: 23.2 kB npm notice unpacked size: 360.1 kB

The error stays the limit is 1MB.

Erkan Sahin 840

Given the package size details you provided, it’s clear that the package is well below the 1MB limit. Here’s a direct approach:

Clear any caches related to npm in Azure DevOps. Sometimes, stale cache can cause unexpected errors.

Ensure the authentication token you’re using has the correct permissions to push to the Azure Artifacts feed.

Check if the Azure Artifacts feed has any custom configurations or policies that might be causing this restriction.

This seems like an anomaly. Reach out directly to Azure support with the error details for a resolution.

Witold Kupś 10 Reputation points

2023-08-23T17:45:43.7566667+00:00

Erkan, I am trying to get the support's help there. Thank you for your attention, but you are not helping, sorry.
Lilei Song 0 Reputation points

2024-07-31T08:54:58.9433333+00:00

Hi, how is the result? I encounter the same issue, and the total file is 4.4 MB, but the Azure throw the error:httpStatusCode: 400, ErrorCode: EntityTooLarge, ErrorMessage: 0:The entity is larger than the maximum allowed size (1MB).

What should I do?

Answer 2

Sumarigo-MSFT 47,466 Microsoft Employee Moderator

@Witold Kupś Thank you for the question. You have posted the query on Azure Storage Service forums, for a specialized assistance on DevOps query, kindly just re-post the question on Developer Community to receive a focused and immediate assistance from the right set of experts. It would the better medium to get better help since the issue doesn't seem to be storage

Please let us know if you have any further queries. I’m happy to assist you further.

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Witold Kupś 10 Reputation points

2023-08-30T04:56:06.35+00:00

I did, the forum below was advised on twitter by your Azure DevOps twitter team
https://techcommunity.microsoft.com/t5/azure-devops/quot-the-package-json-file-is-too-large-size-limit-is-384000/m-p/3912192#M1985
Lilei Song 0 Reputation points

2024-07-31T08:54:34.5133333+00:00

Hi, how is the result? I encounter the same issue, and the total file is 4.4 MB, but the Azure throw the error:httpStatusCode: 400, ErrorCode: EntityTooLarge, ErrorMessage: 0:The entity is larger than the maximum allowed size (1MB).

What should I do?

Answer 3

Lilei Song 0

Hi, how is the result? I encounter the same issue, and the total file is 4.4 MB, but the Azure throw the error:httpStatusCode: 400, ErrorCode: EntityTooLarge, ErrorMessage: 0:The entity is larger than the maximum allowed size (1MB).

What should I do?

Witold Kupś 10 Reputation points

2024-08-02T07:20:10.3633333+00:00

Did not check though, it took just too long since I reported that.
https://developercommunity.visualstudio.com/t/The-packagejson-file-is-too-large-Siz/10461255

Share via

"The package.json file is too large. Size limit is 384000 bytes" when trying to push npm package to azure artifacts

3 answers

Your answer