Large Result Not Chunked (ARCH5062)

Key go.orm.large-result-not-chunked · Severity info · Category orm

Loading an entire large result set into memory then iterating holds millions of rows at once. Process it with a chunked/streaming API.

Incorrect

var users []User
db.Find(&users)
for _, u := range users {
    process(u)
}

Correct

db.FindInBatches(&[]User{}, 1000, func(tx *gorm.DB, batch int) error {
    for _, u := range *tx.Statement.Dest.(*[]User) {
        process(u)
    }
    return nil
})

Fix

Iterate large result sets with a chunked/streaming API like FindInBatches to bound memory use.