MongoDB Cloud Cases: Three Studies

(Khakimullin Aleksandr/Shutterstock)

During the recently MongoDB World conference, Datanami was invited to talk shop with three customers – Braze, Ticketek, and Longbow Advantage – that have implemented the NoSQL database in production cloud environments. Here are their stories:

Ticketek is the largest ticketing company in Australia and New Zealand, processing 20 million tickets annually. “Like Ticketmaster,” says Matt Cudworth, CTO of TEG, Ticketek’s parent company, “but way better.”

Several years ago, the Sydney company embarked upon a major digital transformation strategy, and a key element of that process was selecting a new database management system. Ticketek did what most large companies would do: it had a bake-off.

“We put them all in a row: Oracle, SQL Server, Couchbase, MongoDB,” he says. “Then we replayed an AC/DC concert into these environments.”

In the ticketing business, there are two moments that matter most: The first minute or so when an event goes on sale, and when the guests show up at the venue to have their ticket scanned.

AC/DC demand short-circuited some databases in Ticketek’s stress test

“The only one that got over the hurdle was MongoDB,” Cudworth says. “You could argue that Oracle can do it, but they had to customize it. They didn’t out-of-the-box come anywhere close to what we needed.”

Once the database selection was made, the company got down to business of digital transformation. According to Cudworth, quite a bit of time was spent pulling business logic out of the stored procedures running in its existing Microsoft SQL Server transactional system, and re-building it in MongoDB’s document model.

The company soon turned its internal-focused engineering outward. “We flipped it on its head and said, We have some of these key elements working well. We’re going to turn it to a competitive advantage,” Cudworth says. “We’re going to open the platform, not just technically, but with the mindset that we’re going to build an open ecosystem, a true open platform for the industry.'”

Ticketek discovered that the new system worked so well that it could share data with its customers in real time. “This was the key thig. We almost weaponized the concept of real time data,” Cudworth says. “We walked into key tender and boldly said ‘We’re going to abolish batch processing.'”

MongoDB was working well – particularly with a Redis key-value store providing some caching boost — but the company wasn’t done innovating. To free up Ticketek’s developers’ time, the decision was made to adopt MongoDB’s Atlas cloud service. “I wanted to redeploy the staff he says,” Cudworth says. “I couldn’t manage that many boxes anymore.”

The move to Amazon Web Services brought unexpected advantages, including stricter privacy and security controls. While there was a learning curve to figure out how to work with encrypted databases and encrypted connections, it prepared Ticketek well for the coming regulatory wave, including Australia’s Notifiable Data Breaches act and the Europe Union’s General Data Protection regulation.

“As someone who has run a lot of big workloads in MongoDB, no one should be running off Atlas,” Cudworth says. “[MongoDB CTO] Eliot [Horwitz] said it and he’s right: If you’re serious about what you’re doing, just start with Atlas. You just get so many opportunities and benefits down the track.”

Message in a (Cloud) Bottle

The world is full of messages. There are text messages, email messages, in-app messages, pop-up Web messages, push-messages sent to your phone, and more. It would probably be a stretch to say the world runs on messages, but not by much.

Braze has made a business sending personalized messages on behalf of blue chip names. The New York City company’s messaging business is truly massive: it sends billions of messages per month to customers of clients like Ford, ABC News, and Domino’s Pizza. If you’ve ever received a message from one of these companies, Braze made it happen.

Behind the messaging payload is a complex array of technologies designed to gather, store, and process data about individual customers and their behavior on the Web, in mobile apps, and in physical stores over time. The company says its customer lifecycle engagement platform currently tracks 1.4 billion monthly active users (MAUs), and it’s on track to hit 2 billion MAUs by the end of the year, according to Jon Hyman, the company’s co-founder and CTO.

Hyman, who founded Braze with former Google engineer Bill Magnuson in 2011 cut his teeth on databases while working as an engineer at a Connecticut hedge fund. He was quite familiar with what SQL was capable of, but had an inkling that a relational database wasn’t the right solution for Appboy (which changed its name to Braze in 2017).

“I wanted to try something new,” Hyman says. “At the time I could have [built the platform on a relational database]. But the decision to use MongoDB has greatly simplified some of how our product works.”

The company uses lots of databases, but the core system that stores user data runs across a series of large MongoDB clusters sitting in the cloud. When a client sets up a campaign to automatically send messages to particular groups of customers based on a schedule or real-time events, that criteria triggers database calls to the data stored in MongoDB.

Tracking the metadata associated with send criteria across Braze’s 12 delivery channels is made a simpler task by using a JSON document. “You would need a [relational] table at least for each of them,” Hyman says. “Then if you want to pull a campaign back, that’s eight joins, if a campaign had all those channels, which some of them do.”

What’s nice about Braze’s setup is the company can store all of necessary campaign-related information in one JSON document, and rely on campaign software and the MongoDB database to programmatically gather the right data for each campaign.

“It’s less headache for developers and faster time to iteration,” Hyman says. “We have all the data together. It’s like we have all the joins already. And if we add new embedded relationship because we’re doing something for iOS 12, or if we add Alexa … we’ll have that channel too.”

Braze co-founder and CTO Jon Hyman

MongoDB doesn’t do everything for Braze. The company relies on Redis caches to process 1.5 million operations per second (with peak throughout of 4M OPS). It has more than 1PB of data stored in Apache Hadoop, an S3 data lake, Apache Spark, and analytics software from Arcadia Data, among others. But for core CRUD (create, read, update, delete) operations, “MongoDB is the go-to,” Hyman says.

The company has taken steps to maximize the performance of its MongoDB implementation, which encompasses 1,000 replica sets across 50 clusters and processes on the order of 300,000 read and write operations per second. It currently manages its own MongoDB databases on AWS Elastic Compute Cloud (EC2) machines, but is researching moving it to Atlas, MongoDB’s managed database service.

But where speed is critical, it turns to Rackspace subsidiary ObjectRocket. At Object Rocket, Braze gets bare-metal servers loaded with loads of Flash drives and lots and lots of RAM, all of which makes the MongoDB database fly.

However, speed isn’t the only reason Braze runs on ObjectRocket. “The physical gear has allowed us to have a huge, ultimately private cloud that we can slice and dice,” Hyman says. “Also, being single tenant is a great selling point. We work with financial institutions like Citibank, and we can put them on a single tenant physical machine. So it ends up being great from a customer compliance perspective.”

An Analytics Bullseye

For years, Longbow Advantage focused on implementing warehouse management and labor management solutions from JDA Software, SAP, and HighJump on behalf of customers. The business of automating supply chain operations for big customers was decent, if not growing incredibly fast.

But then the executives noticed something strange: Where were the analytics for supply chain execution?

“What we found was, when you get to the very end of the supply chain in the warehousing and distribution space, there’s not a lot of spend in analytics,” says Alex Wakefield, CEO of Longbow Advantage, which is headquartered in Montreal, Quebec. “Supply chain planning gets the fun stuff, the predictive capabilities. There’s very little money left over for doing distribution analytics.”

In the 2013 timeframe, Wakefield and his team began exploring ways to bring analytics to the warehouse. At the time, the state-of-the-art decision-making device was a big old massive Excel spreadsheet. “These spreadsheets are impressive,” Wakefield admits. “They just take forever to put together.”

The first step was deciding what technology stack to use. By that time, NoSQL had emerged as a viable alternative to the relational databases that had dominated the land for decades. After considering MySQL in the beginning, Longbow quickly narrowed the list down to MongoDB and Couchbase, the two top document NoSQL stores.

In particular, the company liked how the MEAN stack – Mongo, Express.js, Angular JS, and Node.js – fit together, and decided to go for it. “We were comfortable with the size of the MongoDB ecosystem and the technology,” Wakefield said. “When we looked at the selection of the tools, we said ‘Hey this looks like a pretty decent development ecosystem that we can ride on top of.”

After developing the solution for a few years, the company went live with its first client in 2015. The company today has numerous clients for the solution, which is called Rebus Analytics, across various industries. The solution is based on Atlas, MongoDB’s hosted database, and runs in the Google Cloud Platform.

Longbow leverages its deep knowledge of popular ERP and WMS solutions to its advantage. While a plain vanilla BI or analytics vendor would need time to research how to deliver a specific supply chain metric in the dashboard, the Longbow folks know exactly which field in the ERP and WMS databases contains the information.

That’s one of the reasons that the product enjoyed a 4x revenue increase in 2017 and is on pace for a 5x year in 2018, according to Wakefield. “We bring in a low-cost, high-performance solution, and they eat it up,” he says. “We sell to business executives, and they love it.”

Related Items:

Q&A With a MongoDB CTO

MongoDB Fleshes Out Mobile and Cloud Strategies

A MongoDB Secret Weapon: Aggregation Pipeline