marlou knowledge base

# npm memo

  • What’s npm’s mascot? a wombat

  • What does “npm” stand for? Nothing! The original ‘npm’ software was used for Node.js package management

  • How many people does npm employ? npm, Inc. employs 34 people, distributed worldwide

  • What is the largest number of packages that have been downloaded from npm’s Registry in one week? From May 10th to May 17th of 2018, npm users downloaded 5.2 billion packages from the npm Registry, setting a new record.

  • How big is the npm Registry? The registry is approximately 8TB in size, with 25M total package-versions.

# npm 5

Features list

  • better perfs
  • save behavior in package.json when installing a package is reversed (saved by default, flag needed to not save)
  • lockfile strategy changed with package-lock.json generated by default (behavior regarding to the registry is different then shrinkwrap)
  • the deps tree is not displayed after a npm install, print a summary instead
  • complete rewrite of the cache system. npm cache clean is must be run before using npm 5. new command npm cache verify, new options to tweak cache behavior (--prefer-offline, --prefer-online and --offline), concurrent access support.

v5.0.0 - blog.npmjs.org - 20170526

Here’s what you need to know about npm 5 - blog.pusher.com - 20170613

Tout savoir sur la nouvelle version de npm : npm 5 - maxlab.fr - 20170528

npm@5 — Yarn killer? - medium.com - 20170615

# npm 5.1

# npm 5.2

blog.npmjs.org

  • npx added, used to replace npm run-script command. Associated with the new node package, you can run a package against a specific node version.

# npm 5.3

blog.npmjs.org

lots of npx fixes

# npm 5.4

blog.npmjs.org

  • new "did you mean" feature on CLI when making spelling mistakes on the commands
  • npx fixes for Windows platform

# npm 5.5

  • new security features for the registry (2 factors auth, tokens, ...)

# npm 5.6

blog.npmjs.org

  • ability to generate package-lock.json without running npm install
  • fix npm and node9

# npm 5.7

blog.npmjs.org

  • npm install will automatically resolve git merge conflicts in package-lock.json and npm-shrinwrap.json files. cf doc
  • new npm ci command. Install from lock file only and so report errors when a dep is not fixed. node_modules are always rewritten to achieve that.

# npm 5.8

blog.npmjs.org

  • detect-newline (no more LF by default)
  • NO_COLOR standard support through var env
  • NO_PROXY support through CLI with --no-proxy
  • better env var support for .npmrc (cf PR)
  • better error msg for JSON parse error and EPERM/EACCES
  • npm ci fix and perf work

# npm 6

  • npm 6.0.0, le gestionnaire de paquets officiel de Node.js. passe en @latest, et se concentre désormais sur la sécurité
    • npm inc s'offre Lift Security et sa plateforme NSP
    • support natif des hooks npm (port de wombat)
    • intégration des packages create-x dans la commande npm init. Les create-x sont des packages dédiés à la génération d'applications. Par exemple le très connu create-react-app permet de générer un projet de démarrage pour react. En intégrant l'exécution de ce type de package à npm init, cela permet au développeur de partir de la création d'un nouveau projet via npm, donc de configurer son propre package.json tout en déclarant quel package create-x il souhaite utiliser, lequel sera exécuté dans la foulée. Par exemple npm init react-app abouti à la génération de votre package.json puis à l'exécution de create-react-app
    • ajout de la commande npm audit qui comme expliqué dans le paragraphe précédent fournit un audit de sécurité des dépendances de votre projet. Cette fonctionnalité sera pleinement fonctionnelle d'ici quelques semaines, l'application gérant le registre n'ayant pas encore été mise à jour pour supporter cette fonctionnalité. L'exécution de cette commande aboutit pour l'heure à ce message d'erreur : Your configured registry (https://registry.npmjs.org/) does not support audit requests.
    • les versions taguées deprecated des packages ne sont plus installées lorsque c'est possible
    • les commandes npm update et npm outdated tiennent désormais compte du tag latest
    • abandon du support de Node.js 4 et 7
    • diverses corrections de bogues
    • des mises à jour de dépendances

# npm 6.0.1

# npm 6.2.0

# npm 6.3.0

  • v6.3.0
    • npm version now supports a --preid option to specify the preid for prereleases. For example, npm version premajor --preid rc will tag a version like 2.0.0-rc.0

# npm 6.4.0

  • v6.4.0
    • configurable audit level for non-zero exit npm audit currently exits with exit code 1 if any vulnerabilities are found of any level. Add a flag of --audit-level to npm audit to allow it to pass if only vulnerabilities below a certain level are found. Example: npm audit --audit-level=high will exit with 0 if only low or moderate level vulns are detected
    • npm audit now shows no vulnerabilities for npm itself!

# npm 6.5.0

  • v6.5.0
    • Backronym npm ci to npm clean-install

# npm 6.9.0

# npm 6.10.0

  • v6.10.0
    • enable production flag for npm audit : enables using the existing --production flag when running npm audit. Using this flag will ignore dev dependencies when running an audit (see PR)

# npm 6.11.0

  • v6.11.0
    • Implements peerDependenciesMeta(means optional peer deps). See RFC and PR.
    • add new forbidden 403 error code (when requesting a package)

# npm 6.13.0

# npm 6.14.0

# npm 7

  • Release v7.0.0 - 20201013
  • npm v7 Series - Introduction
  • npm v7 Series - Arborist Deep Dive
    • @npmcli/arborist is the dependency tree manager for npm, new in npm v7. It provides facilities for doing nearly everything that npm does with package trees, and fully replaces large parts of the npm CLI codebase.
    • Many of the features and changes in npm v7 are related to the refactor to use Arborist for all of npm’s tree management work.
    • better performance, more predictability, faster feature delivery, and fewer bugs.
    • the core problem, which has led to a lot of excess work and bugfixing in the npm CLI codebase, is that read-package-tree did not properly differentiate between symlinked dependencies and regular installed dependencies, when creating the logical tree of nodes.
    • this new dependencies tree manager opens several new features for npm v7 :
      • a new --prefer-dedupe option is added to tell the tree building algorithm to prefer deduplication over getting the latest version of a dependency.
      • automatic install of peerDependencies (was dropped in npm v4 because of several not resolvable issues).
      • better staging folders management for rollback when npm install fails on Windows platform because of Windows folder locking behaviors.
      • implementation of workspaces.
  • npm v7 Series - Why Keep package-lock.json?
    • if a nutshell, yarn.lock is not fitted for npm.
  • npm v7 Series - Beta Release! And: SemVer-Major Changes in npm v7
    • list all the breaking changes introduced.
    • the bigger is the installation of peerDependencies when running CLI installation commands (npm install, npm ci, etc ...).
    • workspace support added.

# workspaces

Attempt by npm team to have multi-package monorepo built-in support like what is provided by lerna.

See RFC npm workspaces

Add a set of features to the npm cli that provide support to managing multiple packages from within a singular top-level, root package.

See Workspaces in Yarn - classic.yarnpkg.com/blog

# semver

semver to manage scopes in package.json

Warning semver interpretation for breaking changes can be slightly different from a package to another. The semver spec says that any breaking change must be traduced in a ++ on the major version number. But libs like angularjs (or protractor) introduce breaking changes in minor update. Consequences : ^ (caret) in package.json should allow automatic feature update without breaking changes but in fact it depends a lot on the package owner management.

# npm scripts

npm-scripts How npm handles the "scripts" field - docs.npmjs.com

npm supports the "scripts" property of the package.json file, for the following scripts:

  • prepublish: Run BEFORE the package is packed and published, as well as on local npm install without any arguments. (See below)
  • prepare: Run both BEFORE the package is packed and published, on local npm install without any arguments, and when installing Git dependencies (See below). This is run AFTER prepublish, but BEFORE prepublishOnly.
  • prepublishOnly: Run BEFORE the package is prepared and packed, ONLY on npm publish. (See below.)
  • prepack: run BEFORE a tarball is packed (on npm pack, npm publish, and when installing Git dependencies)
  • postpack: Run AFTER the tarball has been generated and moved to its final destination.
  • publish, postpublish: Run AFTER the package is published.
  • preinstall: Run BEFORE the package is installed
  • install, postinstall: Run AFTER the package is installed.
  • preuninstall, uninstall: Run BEFORE the package is uninstalled.
  • postuninstall: Run AFTER the package is uninstalled.
  • preversion: Run BEFORE bumping the package version.
  • version: Run AFTER bumping the package version, but BEFORE commit.
  • postversion: Run AFTER bumping the package version, and AFTER commit.
  • pretest, test, posttest: Run by the npm test command.
  • prestop, stop, poststop: Run by the npm stop command.
  • prestart, start, poststart: Run by the npm start command.
  • prerestart, restart, postrestart: Run by the npm restart command. Note: npm restart will run the stop and start scripts if no restart script is provided.
  • preshrinkwrap, shrinkwrap, postshrinkwrap: Run by the npm shrinkwrap command.

Additionally, arbitrary scripts can be executed by running npm run-script <stage>. Pre and post commands with matching names will be run for those as well (e.g. premyscript, myscript, postmyscript). Scripts from dependencies can be run with npm explore <pkg> -- npm run <stage>.

# command list

# list globally installed package by name
npm -list -g <package_name>

# or for all packages
npm list -g -depth=0

# to get tree deps
npm list -g -depth=1
1
2
3
4
5
6
7
8
# list locally installed package by name
npm -list <package_name>
1
2
# view version on repository
npm view <package_name> version
1
2
# display npm config
npm config list -ls
1
2

11 Simple npm Tricks That Will Knock Your Wombat Socks Off - 20160826

Introducing hooks: get notifications of npm registry and package changes as they happen - 20160601

# npm init + create-* packages

Introduced in npm@6

npm init can now be used to invoke custom scaffolding tools. You can now do things like npm init react-app or npm init esm to scaffold an npm package by running create-react-app and create-esm, respectively.

See 'feat: add npm init <create-pkg-name> command' PR by John Dalton.

See also npm init official documentation.

# npx

npx on www.npmjs.com/package

Executes <command> either from a local node_modules/.bin, or from a central cache, installing any packages needed in order for <command> to run.

By default, npx will check whether <command> exists in $PATH, or in the local project binaries, and execute that. If <command> is not found, it will be installed prior to execution.

Unless a --package option is specified, npx will try to guess the name of the binary to invoke depending on the specifier provided. All package specifiers understood by npm may be used with npx, including Git specifiers, remote tarballs, local directories, or scoped packages.

If a full specifier is included, or if --package is used, npx will always use a freshly-installed, temporary version of the package. This can also be forced with the --ignore-existing flag.

npx added, used to replace npm run-script command.

yarn equivalent of npx ? - yarnpkg/yarn on GitHub - 20170715

zkat (npx maintainer) comment

I'm working on library-ifying npx. It's not a huge task to grab the existing npx code and just replace the npm-related guts with the yarn-equivalent commands.

I won't add that directly to npx itself, since it's meant to be agnostic: npx performs no operations which clash with people using other package managers. It doesn't even require npm to be on the system, so you can npm rm -g npm and npx will work just fine. So you could say npx is ypx, unless you feel really strongly about cache-sharing, which is a pretty thing.

(in re inspiration: npx is primarily inspired by this long-standing feature request: npm/npm#6053. Most of its functionality centers around fulfilling this need. The auto-install feature was added post-yarn-create, and is definitely intended to be an actual generalized solution to that particular thing -- but it does way more than that)

# testing packages locally

The magic behind npm link

Node.js — How to test your new NPM module without publishing it every 5 minutes - updated 201804

Testing NPM alpha / beta / rc packages - 20180508

3 options to require a your package in your hosting project :

  • npm link

It will symlink your project in the global node_modules directory.

So it pollutes the global npm namespace.

No preinstall/postinstall hooks will be triggered so if you need to tests them it's not a good choice.

npm link <dep-name> will not alter the package.json file of the hosting project.

In your dependency directory :

npm link # create a global symlink to the local "dependency-name" project
1

In your hosting project :

npm link dependency-name # create a symlink locally to global dependency-name
1
  • npm install /absolute/local/path

It works as if your package was is the npm registry. But it will write the absolute directory path to reach the dependency in the package.json of your hosting project :

npm install /absolute/path/to/dependency-name
1

yields this in our package.json :

"dependencies": {
  "dependency-name": "file:../../projects/dependency-name",
},
1
2
3
"dependencies": {
  "viking": "file:../../oresoftware/viking",
},
1
2
3
  • npm pack

The npm pack command create the tarball that will be pushed in the registry.

So we can build the payload and test against it before pushing it to the registry.

Theses kind of tests are named smoke-tests (cf discussion on stackoverflow).

Don't forget to use a .npmignore file (cf npm doc) to avoid putting editor config files (.idea/ for ex) in the tarball. Notice that npm publish run npm pack so ignore .tgz files to avoid adding it to the uploaded package.

In your dependency directory :

npm pack # create the tarball in the root dep project directory
1

In your hosting project you can now install it like a regular package, there is no difference instead that :

  • it will not be downloaded from the network but from your local filesystem
  • there will be also an absolute path in your host project package.json file

# npmc testing @next npm version

npm versions tagued @next can be tested without upgrading your current npm version with the npmc package.

c is for canary.

# npm folders

check this

TLDR :

  • Local install (default): puts stuff in ./node_modules of the current package root.
  • Global install (with -g): puts stuff in /usr/local or wherever node is installed.
  • Install it locally if you're going to require() it.
  • Install it globally if you're going to run it on the command line.
  • If you need both, then install it in both places, or use npm link.

# managing dependencies

# built-in commands

npm ls

# list all the installed dependencies in stdout in a tree format
npm ls

# list all the installed dependencies in stdout in a tree format limited to the first level
# it must matches deps listed in package.json
npm ls --depth=0

# same as above but with the deps of deps in the tree
npm ls --depth=1

# print extended informations
npm ll
npm la
1
2
3
4
5
6
7
8
9
10
11
12
13

npm outdated

# check the registry to see if any (or, specific) installed packages are currently outdated
npm outdated
1
2

# fixing deps

Since npm v5, the lockfile standard is no more shrinkwrap, it is now the package-lock.json file.

The big difference is that this file is generated automatically by npm when running the install command.

There is also big behavior differences regarding to the registry publication for packages.

npm-shrinkwrap.json is backwards-compatible with npm versions 2, 3, and 4, whereas package-lock.json is only recognized by npm 5+

What is the difference between npm-shrinkwrap.json and package-lock.json? - stackoverflow.com - 20170530

# understanding peer dependencies

docs.npmjs.com - package.json - peerdependencies

you want to express the compatibility of your package with a host tool or library, while not necessarily doing a require of this host

{
  "name": "tea-latte",
  "version": "1.3.5",
  "peerDependencies": {
    "tea": "2.x"
  }
}
1
2
3
4
5
6
7

This ensures your package tea-latte can be installed along with the second major version of the host package tea only.

This leads to

├── tea-latte@1.3.5
└── tea@2.2.0
1
2

instead of

├── tea-latte@1.3.5
   └── tea@2.2.0
1
2

since npm@3, npm is not installing automatically peerDeps. You need to add it explicitly in your package.json.

# peerDeps usage

Mostly for a package to be used as a deps for another. No usage for an end project (like a frontend or a backend).

# peerDeps tools

nathanhleung/install-peerdeps - github.com

A command-line interface to install an NPM package and its peer dependencies automatically.

# others packages

Check for outdated, incorrect, and unused dependencies.

Find newer versions of dependencies than what your package.json or bower.json allows

Examine a package's dependency graph before you install it

← npm - windows →