Interactive load gets all the attention, but in practice a lot of CRM slowness comes from work nobody is watching: the nightly sync, the list import someone started at 10 a.m., the workflow that recalculates scores for every contact after a rule change. These jobs share the same database and servers as your users, and when they misbehave, everyone feels it.

Scaling this side of the system is about shaping the work so it runs steadily without crowding out people.

The three kinds of background work

Separating them helps, because each wants different treatment.

• Scheduled jobs: recurring tasks like nightly recalculations, deduplication runs, report generation and exports.
• Event-triggered jobs: work that follows a user action or an incoming message, like sending a notification after a deal closes or enriching a new lead.
• Bulk operations: large one-off or periodic loads such as imports, mass updates and migrations.

A common trap is treating all three as the same code path. A bulk import that goes through the same per-record triggers and workflows as a manual edit can set off a chain reaction: every imported record fires an email rule, a scoring rule and an outbound sync, multiplied by thousands.

Put a queue in the middle

The most useful structural change is decoupling the request from the work. Instead of doing the work while the caller waits, record the intent on a queue and let workers pick it up.

Benefits you get:

• Bursts become backlog instead of outage. A sudden flood of events sits in the queue and drains at a rate the system can sustain.
• You control concurrency. Ten workers means at most ten things happening at once, whatever the incoming rate.
• Failures become retryable. A failed job returns to the queue with a delay instead of being lost.
• Capacity scales independently. Add workers when the backlog grows, without scaling the web tier.

Things you must handle in return: jobs must be safe to run twice (idempotent), repeatedly failing jobs need a dead-letter holding area plus an alert, and ordering rules need thought if two jobs touch the same record.

Many CRM platforms provide their own job frameworks with built-in limits. Check the vendor’s current documentation for how asynchronous processing, batch sizes and concurrency are governed before building your own.

Batch sensibly

Row-by-row processing is slow because each operation pays overhead. One giant transaction is also bad, since it holds locks and can roll back an hour of work on a single bad record. The sweet spot is moderate batches:

• Process records in chunks that complete quickly.
• Commit after each chunk so progress is saved.
• Record a checkpoint so a failed run resumes where it stopped rather than restarting.
• Handle bad records by setting them aside with a reason, not by aborting everything.

Pick chunk size by testing against realistic data. The right number depends on record width, triggers and database behaviour, so measure it.

Throttle and schedule

Even well-behaved jobs need limits. Give heavy work a leash:

1. Rate limits on API calls to downstream systems, so you do not trigger their throttling and fill your retry queue.
2. Priority lanes. Customer-visible work, like sending a quote, should not wait behind a deduplication run. Separate queues or priorities solve this.
3. Time windows. Run heavy recalculations overnight or at weekends in the users’ time zones, and pause them during known peaks such as month end.
4. Concurrency caps per job type, so one runaway job cannot consume every worker.
5. Back-pressure. If the database latency rises past a threshold, workers slow down or pause automatically.

Controlling trigger storms

Before a big import, ask what each inserted record will trigger. List the workflows, validation rules, duplicate checks, notifications and integrations attached to the object. Where the platform allows it, disable non-essential automation for the duration of the load and run the necessary follow-up steps afterward in a controlled batch. Never disable outbound customer communications blindly; decide explicitly what should and should not fire.

What to monitor

Signal	What it tells you	Typical response
Queue depth growing steadily	Workers cannot keep up with arrivals	Add workers or reduce intake
Oldest job age	Real delay seen by users	Alert when it passes the agreed limit
Failure and retry rate	Bad data or a downstream problem	Inspect dead-letter items
Job duration trend	Data growth or a query regression	Investigate when it drifts upward
Database latency during jobs	Jobs crowding users	Throttle or reschedule
Run overlap	A job still running when the next starts	Add locking or lengthen the interval

Overlap deserves a special mention. A job scheduled hourly that sometimes takes ninety minutes will pile on itself unless it checks for an existing run.

A hypothetical

Consider a team that imports a large partner contact list each quarter during working hours. Every import slows the CRM for the sales team and sets off several hundred duplicate-warning emails. They change the process: the import runs through a queue with a modest concurrency cap, in batches with checkpoints, scheduled for the evening. Notification rules are paused for the import and a single summary email is sent at the end. Records that fail validation go to a review file instead of stopping the run.

The first run of the new process takes longer overall than the old reckless one, and nobody notices because it happens overnight. That trade is almost always right.

Frequently Asked Questions

Should every CRM job go through a queue? Not necessarily. Short, fast, user-initiated operations are fine inline. Anything slow, bursty, retryable or large belongs on a queue.

How do I stop a job from running twice at once? Use a lock or lease so only one instance of a given job runs, and make the work idempotent so an accidental rerun does no harm.

What batch size should I use? There is no universal number. Start small, measure throughput and lock behaviour on realistic data, and increase until the benefit flattens or user impact appears.

Can I run imports during the day? You can if they are throttled and monitored, but off-peak is safer. Large mass updates in particular are better after hours.

How do I handle failures in the middle of a bulk load? Use checkpoints so you can resume, set aside bad records with reasons, and design the load so rerunning it does not duplicate data.

Next Step

List every scheduled job and recurring import in your CRM, with its start time, usual duration and owner. Look for overlaps with business hours or with each other, and move the single heaviest one to a quieter slot this week.