Understanding npm Folder Structure for Local and Global Installs

The last tutorial concluded a series on npm's CLI command, where we should you how to work with npm from the cli.

This tutorial will start a new series on configuring npm, here we will show you where npm puts the various thing you need in your environment.

Npm-folders

This is the folder structure used by npm.

Description

The job of npm is to collect lots of things into your computer, these things include packages and their dependencies.

This tutorial will tell you where npm puts these various things.

tl;dr

• Local install (default): this will put stuff in ./node_modules of the current package root.
• Global install (with -g): this will put stuff in /usr/local or wherever you installed node.
• You should install it locally if you are going to require() it.
• You should install it globally if you are going to run it on the command line.
• If you need both, then you should install it in both places, or use npm link.

prefix Configuration

The prefix config will default to the location where node is installed. This is /usr/local for most systems. For Windows, it is %AppData%\npm. While for Unix systems, it is one level up, this is because node is typically installed at {prefix}/bin/node rather than {prefix}/node.exe.

When you set the global flag, npm will install things into this prefix. When you do not set this, it will use the root of the current package, or the current working directory if you are not in a package already.

Node Modules

All packages are dropped into the node_modules folder under the prefix, when installing locally. This means that you can require("packagename") to load its main module, or you can require("packagename/lib/path/to/sub/module") to load other modules.

Global installs on Unix systems are stored in {prefix}/lib/node_modules. While the global installs on Windows go to {prefix}/node_modules (that is, no lib folder.)

Scoped packages are installed in the same way, except they will be grouped together in a sub-folder of the relevant node_modules folder with the name of that scope prefix by an @ symbol, e.g. npm install @myorg/package places the package in {prefix}/node_modules/@myorg/package.

If you want to require() a package, then you should install it locally.

Executables

When you are in global mode, executables will be linked into {prefix}/bin on Unix, or directly into {prefix} if you are using Windows.

When in local mode, executables will be linked into ./node_modules/.bin so that they can be made available to scripts that are run through npm.

Man Pages

Whenever you are in global mode, man pages will be linked into {prefix}/share/man.

However, man pages are not installed when you are in local mode.

It should be noted that man pages are not installed on Windows systems.

Cache>

Cache files are stored inside ~/.npm on Posix, or %AppData%/npm-cache for Windows.

The cache is controlled by the cache configuration param.

Temp Files

All temporary files are stored by default in the folder specified by the tmp config, this defaults to the TMPDIR, TMP, or TEMP environment variables, or /tmp for Unix or c:\windows\temp for Windows.

Temp files will always be given a unique folder under this root for each run of the program, and will be deleted upon successful exit.

More Information

Whenever installing locally, npm first will try to find an appropriate prefix folder.

Starting at the $PWD, npm walks up the folder tree checking for a folder that contains either a package.json file, or a node_modules folder. If such a folder is found, then it is treated as the effective "current directory" for the purpose of running npm commands.

If it does not find a package root, then the current folder will be used.

Whenever you run npm install [email protected], then the package will be loaded into the cache, and then it will be unpacked into ./node_modules/foo. Then, any of foo's dependencies will similarly be unpacked into ./node_modules/foo/node_modules/

All bin files will be symlinked to ./node_modules/.bin/, so that they can be found by npm scripts when necessary.

Global Installation

If you set the global configuration to true, then npm installs packages "globally".

For global installation, packages will be installed roughly the same way, but will use the folders described above.

Cycles, Conflicts, and Folder Parsimony

Cycles are always handled using the property of node's module system that it walks up the directories searching for node_modules folders. So, at every stage, if a package is installed already in an ancestor node_modules folder, then it will not be installed at the current location.

Consider the case where foo -> bar -> baz. Imagine if, baz also depends on bar, so you would have: foo -> bar -> baz -> bar -> baz .... However, since the folder structure is this: foo/node_modules/bar/node_modules/baz, there is no need to put another copy of bar into .../baz/node_modules, because whenever it calls require("bar"), it gets the copy that is installed in foo/node_modules/bar.

This shortcut will only be used if the exact same version will be installed in multiple nested node_modules folders. It is still possiblefor you to have a/node_modules/b/node_modules/a if the two "a" packages are different versions. However, without repeating the exact same package multiple times, an infinite regress is always prevented.

Another optimization can be made by installing the dependencies at the highest level possible, the localized "target" folder is shown below.

Example:

Consider this dependency graph:

foo
+-- [email protected]
+-- [email protected]
|   +-- [email protected] (latest=1.3.7)
|   +-- [email protected]
|   |   `-- [email protected]
|   |       `-- [email protected] (cycle)
|   `-- asdf@*
`-- [email protected]
    `-- [email protected]
        `-- bar

In this case, we might expect a folder structure like this:

foo
+-- node_modules
    +-- blerg (1.2.5) <---[A]
    +-- bar (1.2.3) <---[B]
    |   `-- node_modules
    |       +-- baz (2.0.2) <---[C]
    |       |   `-- node_modules
    |       |       `-- quux (3.2.0)
    |       `-- asdf (2.3.4)
    `-- baz (1.2.3) <---[D]
        `-- node_modules
            `-- quux (3.2.0) <---[E]

Since the foo package depends directly on [email protected] and [email protected], those will be installed in foo's node_modules folder.

Although the latest copy of blerg is 1.3.7, foo specifically depends on version 1.2.5. Hence, that is what is installed at [A]. Also the parent installation of blerg satisfies bar's dependency on [email protected], hence it does not install another copy under [B].

Bar [B] equally has dependencies on baz and asdf, hence those are installed in bar's node_modules folder. Since it depends on [email protected], it cannot re-use the [email protected] that is installed in the parent node_modules folder [D], and thus must install its own copy [C].

Below bar, the baz -> quux -> bar dependency will create a cycle. However, since bar is already in quux's ancestry [B], it doesn't unpack another copy of bar into that folder.

Below foo -> baz [D], the quux's [E] folder tree is empty, since its dependency on bar is satisfied by the parent folder copy installed at [B]. To see a graphical breakdown of what is installed where, run npm ls.

Publishing

Upon publishing, npm looks in the node_modules folder. Any of the items that is not in the bundledDependencies array, will not be included in the package tarball.

This will allow a package maintainer to install all of their dependencies (and dev dependencies) locally, but will only re-publish those items that cannot be found elsewhere.

Previous: How to View Registry Information and Display Username Using npm CLI.
Next: Managing npm Configuration Files with npmrc.