Comprehensive Guide to Using db.collection.insertMany in MongoDB

Understanding db.collection.insertMany() in MongoDB

The db.collection.insertMany() method in MongoDB allows the insertion of multiple documents into a collection in a single operation. This method is highly efficient and ensures all documents are written in one transaction-like operation, making it ideal for bulk inserts.

Syntax:

db.collection.insertMany(
   [ <document 1> , <document 2>, ... ],
   {
      writeConcern: <document>,
      ordered: <boolean>
   }
) 

Parameters:

Compatibility:

You can use db.collection.insertMany() for deployments hosted in the following environments:

• MongoDB Atlas: The fully managed service for MongoDB deployments in the cloud
• MongoDB Enterprise: The subscription-based, self-managed version of MongoDB
• MongoDB Community: The source-available, free-to-use, and self-managed version of MongoDB

Behaviors:

Given an array of documents, insertMany() inserts each document in the array into the collection.

Execution of Operations

• By default, documents are inserted in the order they are provided.
• If ordered is set to true and an insert fails, the server does not continue inserting records.
• If ordered is set to false and an insert fails, the server continues inserting records. Documents may be reordered by mongod to increase performance. Applications should not depend on ordering of inserts if using an unordered insertMany().
• The number of operations in each group cannot exceed the value of the maxWriteBatchSize of the database. The default value of maxWriteBatchSize is 100,000. This value is shown in the hello.maxWriteBatchSize field.
• This limit prevents issues with oversized error messages. If a group exceeds this limit, the client driver divides the group into smaller groups with counts less than or equal to the value of the limit. For example, with the maxWriteBatchSize value of 100,000, if the queue consists of 200,000 operations, the driver creates 2 groups, each with 100,000 operations.

Note:

The driver only divides the group into smaller groups when using the high-level API. If using db.runCommand() directly (for example, when writing a driver), MongoDB throws an error when attempting to execute a write batch which exceeds the limit.

If the error report for a single batch grows too large, MongoDB truncates all remaining error messages to the empty string. If there are at least two error messages with total size greater than 1MB, they are trucated.

The sizes and grouping mechanics are internal performance details and are subject to change in future versions.

Executing an ordered list of operations on a sharded collection will generally be slower than executing an unordered list since with an ordered list, each operation must wait for the previous operation to finish.

Collection Creation

If the collection does not exist, then insertMany() creates the collection on successful write.

_id Field

• If the document does not specify an _id field, then mongod adds the _id field and assign a unique ObjectId() for the document. Most drivers create an ObjectId and insert the _id field, but the mongod will create and populate the _id if the driver or application does not.
• If the document contains an _id field, the _id value must be unique within the collection to avoid duplicate key error.

Explainability

insertMany() is not compatible with db.collection.explain().

Error Handling

Inserts throw a BulkWriteError exception.

• Excluding write concern errors, ordered operations stop after an error, while unordered operations continue to process any remaining write operations in the queue.
• Write concern errors are displayed in the writeConcernErrors field, while all other errors are displayed in the writeErrors field. If an error is encountered, the number of successful write operations are displayed instead of a list of inserted _ids. Ordered operations display the single error encountered while unordered operations display each error in an array.
• Schema Validation Errors

If your collection uses schema validation and has validationAction set to error, inserting an invalid document with db.collection.insertMany() throws a writeError. Documents that precede the invalid document in the documents array are written to the collection. The value of the ordered field determines if the remaining valid documents are inserted.

Transactions

db.collection.insertMany() can be used inside distributed transactions.

Collection Creation in Transactions

You can create collections and indexes inside a distributed transaction if the transaction is not a cross-shard write transaction.

If you specify an insert on a non-existing collection in a transaction, MongoDB creates the collection implicitly.

Write Concerns and Transactions

Do not explicitly set the write concern for the operation if run in a transaction. To use write concern with transactions, see Transactions and Write Concern.

Example Usage

Basic Example

Code:

// Insert multiple documents into the 'products' collection
db.products.insertMany([
  { name: "Laptop", price: 1200, stock: 30 },  // First document
  { name: "Mouse", price: 25, stock: 100 },   // Second document
  { name: "Keyboard", price: 45, stock: 50 }  // Third document
])

Explanation:

• Three documents are added to the products collection.
• If no _id is provided, MongoDB automatically generates one for each document.

Insert with Custom _id Values

Code:

// Insert multiple documents with custom _id values
db.orders.insertMany([
  { _id: 1, item: "Phone", qty: 10 },
  { _id: 2, item: "Tablet", qty: 5 },
  { _id: 3, item: "Charger", qty: 20 }
])

Explanation:

• Custom _id values (1, 2, 3) are specified for each document.
• MongoDB will not generate _id fields since they are explicitly provided.

Insert with Options

Code:

// Insert multiple documents with options
db.books.insertMany(
  [
    { title: "Book A", author: "Author 1" },
    { title: "Book B", author: "Author 2" },
    { title: "Book C", author: "Author 3" }
  ],
  { ordered: false }
)

Explanation:

• The ordered: false option allows MongoDB to continue inserting documents even if one fails.

Returned Acknowledgment

When db.collection.insertMany() executes successfully, it returns an acknowledgment object:

Code:

{
  "acknowledged": true,
  "insertedIds": {
    "0": ObjectId("648ec9b1f7e8f0001a2d7e8a"),
    "1": ObjectId("648ec9b1f7e8f0001a2d7e8b"),
    "2": ObjectId("648ec9b1f7e8f0001a2d7e8c")
  }
}

Explanation:

• acknowledged: Indicates the operation was successful.
• insertedIds: A mapping of array indexes to the _id values of the inserted documents.

Error Scenarios

Duplicate _id Error

Code:

// Attempting to insert documents with duplicate _id values
db.orders.insertMany([
  { _id: 1, item: "Laptop", qty: 10 },
  { _id: 1, item: "Tablet", qty: 5 } // Duplicate _id
])

Error:

Code:

E11000 duplicate key error collection: test.orders index: _id_ dup key: { _id: 1 }

Reason:

• The _id value 1 is already present in the collection.
• MongoDB will halt further inserts if ordered: true is set (default behavior).

Invalid Document Structure

Code:

// Invalid data type for a document
db.products.insertMany([
  { name: "Phone", price: 799 },
  "InvalidDocument" // Not a valid JSON object
])

Error:

Code:

TypeError: Document must be a valid JSON object

Reason:

• All elements in the documents array must be valid JSON objects.

Best Practices

1. Use Ordered Inserts Only When Necessary:

• Set ordered: false for bulk inserts to avoid complete failure if one document fails.

2. Validate Data Before Insert:

• Use validation scripts or MongoDB schema validation to ensure data integrity.

3. Avoid Duplicate _id Values:

• Ensure _id values are unique when manually specified.

4. Monitor Bulk Insert Performance:

• For large datasets, consider breaking inserts into batches to avoid memory issues.

Additional Examples

Inserting Large Data Sets

Code:

// Insert a large array of documents using insertMany
const largeData = [];
for (let i = 0; i < 1000; i++) {
  largeData.push({ item: `Item ${i}`, stock: Math.floor(Math.random() * 100) });
}
db.inventory.insertMany(largeData);

Explanation:

• A loop creates 1000 unique documents and inserts them into the inventory collection.

Troubleshooting Tips

1. Duplicate Key Issues

• Use the ordered: false option to bypass errors and insert remaining documents.

2. Insert Performance

• Ensure sufficient memory and disk space for large batch inserts.

3. Connection Timeout

• For remote databases, increase the timeout setting in the MongoDB driver.