How we Effectively Scaled the Contact Insights Computation From 0 orgs to 20k orgs With our Spark Data Pipeline

How we Effectively Scaled the Contact Insights Computation From 0 orgs to 20k orgs With our Spark Data Pipeline

In the world of active conversation across multiple sales reps and customers, there is always a case that a sales rep needs a quick introduction to kickstart their sales process. With millions of conversations going around across multiple user base, building activity graph is a time consuming operation. The scale for computation becomes harder when we need to consistently compute for 20k organizations, and keep the closest computations updated and better with latest conversations and newer relations. We are walk through with our initial approach of solving this harder scale problem, different approaches we choose and fail, and how we effectively scaled it up for growing number of orgs.

Schedule
Room
Ballroom B
Speakers
Praveen
Innamuri
Sr Engineering Manager
at
Salesforce
I've been very passionate about distributed systems & data engineering, and I run a couple of engineering teams at Salesforce in building data ingestion pipelines and contact insights generation.
Zhidong
Ke
Senior Software Engineer
at
Salesforce
Zhidong has a historical background with running and scaling data infra jobs all the way from hadoop to spark along with various real time engines such as storm.

Slides & Recordings

   Download Slides