Node.JS (JavaScript / TypeScript) dependencies, known as npm packages.
See the hosting-npm lab directory for research and progress pertaining to this feature.
I now have the ability to generate a holistic dependency tree cache using both npm and pnpm, and can subsequently install offline via the cache. I can also optionally pack the tarballs for each dependency in the tree.
What I have yet to figure out is how to properly establish a central registry that npm can then be pointed to via .npmrc.
I initially investigated using verdaccio to serve as an offline dependency registry. Out of the box, it serves as a registry proxy to https://registry.npmjs.org/ and interacts with the local cache in the same way as I am building it in the above scripts.
Relying on the npm cache is inherently not ideal because anytime you need to add packages to it, you cannot cherry pick the new dependencies. You must rebuild the cache as persistence in the cache is unreliable at best. It seems promising if there ends up being no other alternatives, but there is more investigation to be done. I'm a little hesitant to rely on another third-party product.
What would be preferrable would be to establish an HTTP server that is appropriately structured to host the dependencies in their proper structure and format. This would facilitate being able to maintain a centralized, persistent npm registry that can be updated with only new artifacts with each passing update session. See npm registry and CommonJS Package Registry for more details.
To figure out:
registry value can be used for native retrieval via npm i <package> and pointing the config to the registry.The PowerShell script Build-NpmCache.ps1 defines the ability to generate npm projects with locally cached packages. The generated projects can then be transported to a disconnected network and used to establish new projects or update the dependencies for an existing Node.js project.
The resulting directory structure for each project should be:
The PowerShell script Build-NpmCache.ps1 defines the ability to generate a cache of npm packages, as well as any associated binaries, intended for global installation. Additionally, you can set environment variables that will be initialized for the duration of the global cache generation. This cache can then be transported to a disconnected network and used to establish or update globally installed npm packages.
A good example of a global npm package that uses all of these features is Cypress. When cypress is globally installed, it automatically installs the associated binary data in the cypress cache at $env:LocalAppData\Cypress\Cache on windows, and ~/.cache/Cypress on linux.
Cypress provides a series of environment variables that can be set to control the way it behaves when installed. Notably, the CYPRESS_INSTALL_BINARY variable can be set to 0 to indicate that the binary should not be automatically installed when installing the npm package.
Additionally, binaries can be downloaded using the provided Download URLs provided by cypress. Instead of downloading the binaries to the local user cache when caching the cypress package, the binary will automatically be downloaded based on the metadata provided in the binaries array of the configuration passed to the script:
"global": {
// cache directory for global npm packages
"target": "global",
/*
list of environment variables to set
while generating the global npm cache
*/
"environment": [
{
// the environment variable to set
"key": "CYPRESS_INSTALL_BINARY",
// the environment variable value
"value": 0
}
],
// the global npm packages to cache
"packages": [
"cypress"
],
/*
list of external binaries associated with
the npm packages being cached
*/
"binaries": [
{
// cache directory for the binary
"target": "cypress_cache",
// file name for the downloaded binary
"file": "cypress.zip",
// download URI for the binary
"source": "https://download.cypress.io/desktop?platform=win32&arch=x64"
}
]
}
A sample project for this capability can be found at /lab/local-npm.
In the npm install docs define a package as:
<name>@<version> that is published on the registry (see registry) with (c)<name>@<tag> (see npm dist-tag) that points to (d)<name> that has a latest tag satisfying (e)<git remote url> that resolves to (a)The sections that follow will walk through how to scaffold a local TypeScript package that satisfies (a) above, then install and consume the local package in a Node.js project.
Initialize a Node.js project
# create the root package directory
mkdir lib
# change directory to the package
cd ./lib/
# initialize the Node.js project
npm init
Fill in the npm init details:
{
"name": "@local/simple-storage",
"version": "0.0.1",
"description": "Abstraction layer for interfacing with browser storage",
"main": "dist/index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "Jaime Still",
"license": "MIT"
}
Add dependencies:
# dependencies
npm i uuid
# dev dependencies
npm i -D @types/uuid typescript
Adjust package.json with the following:
{
"types": "dist/index.d.ts",
"scripts": {
"build": "tsc",
"watch": "tsc --watch"
}
}
Create a tsconfig.json:
{
"compileOnSave": false,
"compilerOptions": {
"moduleResolution": "node",
"target": "ES2022",
"module": "ES2022",
"rootDir": "./src",
"outDir": "./dist",
"declaration": true,
"esModuleInterop": false,
"forceConsistentCasingInFileNames": true,
"strict": true
}
}
Create a ./src directory, build out your TypeScript files, and export your public API in index.ts:
export * from './base-storage'
export * from './istorage'
export * from './local-storage'
export * from './session-storage'
In a Node.js project, install the package as a dependency:
npm i ../lib
package.json should now contain a reference:
{
"dependencies": {
"@local/simple-storage": "file:../lib"
}
}
Use the package in your project:
import {
IStorage,
SessionStorage
} from '@local/simple-storage';
store: IStorage<string> = new SessionStorage();