SharePoint Online to Online Migration – PnP PowerShell Batch Update for System Fields

[I-BhushanSalunkhe]

We are performing a SharePoint Online to SharePoint Online migration using PnP PowerShell.

Due to new functional requirements, we redesigned the architecture:

New architecture

• Global master data is stored at the site collection level.
• Transactional data is stored within respective subsites.
• Subsite-specific master data is also maintained within each subsite.

Existing architecture

• All data (masters + transactional) resides at the site collection level.

We developed a PnP PowerShell script that:

• Creates list items in the respective subsite (including relational data).
• Updates system fields such as:
  Created
Author
Modified
Editor

The script works correctly when executed sequentially. However, due to the large data volume, the execution time is very long.

Attempt to Optimize Using Batch

To improve performance, we attempted to use batching:

$batch = New-PnPBatch

foreach ($item in $items) {

    Add-PnPListItem -List $listName -Values @{
        "Title"  = $item.Title
        "Status" = $item.Status
    } -Batch $batch

    Set-PnPListItem -List $listName -Identity $item.Title -Values @{
        "Created"  = $item.FieldValues["Created"]
        "Modified" = $item.FieldValues["Modified"]
        "Author"   = $item.FieldValues["Author"].Email
        "Editor"   = $item.FieldValues["Editor"].Email
    } -Batch $batch
}

Invoke-PnPBatch -Batch $batch

Issue

Since batched operations are executed only when Invoke-PnPBatch is called, the items do not yet exist when Set-PnPListItem is queued. Therefore, we are unable to update the system fields in the same batch.

Question

What is the recommended approach to:

• Create items in batch for performance optimization
• Update system fields (Created, Author, Modified, Editor)
• While maintaining relational integrity
• And handling large data volumes efficiently?

Any guidance or best practices for this scenario would be highly appreciated.

For more details on migration, please refer to this bug we reported.