Spreadshirt
Spreadshirt Platform
An Architectural Overview
October 2016
Jens Hadlich, Chief Architect
Spreadshirt
About Spreadshirt
2
Spread it with Spreadshirt
	A	global	e-commerce	pla.orm	for	everyone	to	create,	sell	and	buy	ideas	on	
clothing	and	accessories	across	many	points	of	sale.	
•  Founded	in	2002,	based	in	Leipzig	
•  550+	employees	around	the	world	
•  Global	shipping	(180+	countries)	
•  Community	of	70.000+	acFve	sellers	
•  €	85M	revenue	(2015)	
•  3.6M	shipped	items	(2015)
Spreadshirt
Spreadshirt Platform
3
Apps (iOS, Android)
Marketplace
Blogs
Integrations &
Custom Solutions
Video & photo
External Fulfillment
T-Shirt Designer
Spreadshirt
Platform
API
External Marketplaces
Social Media
Shops
Spreadshirt 4
Some Numbers
Spreadshirt
Spreadshirt Platform in Numbers
5
300 Million
Page Impressions
per month, globally
25 Thousand
DB Queries
per second, daily average
>1 Terabyte*
DB Storage
>25 Terabyte*
File Storage
images / raw data
* without backup copies, replicas etc.
Spreadshirt 6
Platform Architecture
Spreadshirt
Platform Architecture
7
Do you like furry animals?
Spreadshirt
Platform Architecture
8
Do you like furry animals?
“The” Opossum
Spreadshirt
The Past
9
“One system to rule them all.”
Opossum
CMS
Checkout
Payments
Admin
Tracking
Orders
…
Shops User Area
Website
Marketplace Designer
Spreadshirt
NOpossum
10
Spreadshirt
From Monolith to Microservices
11
Spreadshirt
From PHP to Java
12
Spreadshirt
Platform Architecture Evolution
13
Opossum
CMS
Checkout
Payments
Admin
Tracking
Orders
…
Shops User Area
Website
Marketplace Designer
Spreadshirt
Platform Architecture Evolution
14
OpossumAPI
Data API
Image API
CMS
Checkout
Payments
Admin
Tracking
Orders
…
Shops User Area
Website
Marketplace Designer
Search
Spreadshirt
Platform Architecture Evolution
15
Message Bus
OpossumAPI
Data API
CMS
Router
Admin
Tracking
…
Shops User Area
Website
Marketplace Designer
Image API Search
Checkout
Payments
Orders
Spreadshirt
Platform Architecture Evolution
16
Message Bus
OpossumAPI
Data API
CMS
Router
Admin
Tracking
* API-based ** “good old” Shop2
…
Shops** User Area
WebsiteMarketplaceShops*Designer
Image API Search
Checkout
Payments
Orders
Spreadshirt
Platform Architecture Evolution
17
Message Bus
OpossumAPI
Data API
CMS
Router
Admin
Tracking
Checkout
* API-based ** “good old” Shop2
MarketplaceShops*
…
Designer
Shops** User Area
Website
Image API Search
Checkout
Payments
Orders
Spreadshirt
Platform Architecture Evolution
18
Message Bus
OpossumAPI
Data API
Image API
CMS
Router
Admin
Tracking
Checkout
* API-based ** “good old” Shop2
UMP
Tracking
RankingSearch
MarketplaceShops*Designer
Shops** User Area
Website
…
Payments
Orders
Spreadshirt
Platform Architecture Evolution
19
Message Bus
OpossumAPI
Data API
Image API
CMS
Router
Google Merchant Feed
EMP Service
Admin
...
Amazon
EMP Tools
Production ERP Newsletter …
Checkout
* API-based ** “good old” Shop2
UMP
Tracking
RankingSearch
MarketplaceShops*
Partner Area
DesignerDesignerDesigner
Shops** User Area
Website
Payments
Orders
Spreadshirt
Technologies
2020	
Front End Back End Infrastructure
Spreadshirt
Platform Architecture & Technologies
21
Message Bus
OpossumAPI
Data API
Image API
CMS
Router
Google Merchant Feed
EMP Service
Admin
...
Amazon
EMP Tools
Production ERP Newsletter …
Checkout
* API-based ** “good old” Shop2
UMP
Tracking
RankingSearch
MarketplaceShops*
Partner Area
DesignerDesignerDesigner
Shops** User Area
Website
Payments
Orders
Spreadshirt
Platform Architecture
22
The reality …
Spreadshirt
Platform Architecture
23
… is a bit more complicated
The reality …
Spreadshirt 24
Going Global
Spreadshirt
Platform Architecture
25
Message Bus
OpossumAPI
Data API
Image API
CMS
Router
Google Merchant Feed
EMP Service
Admin
...
Amazon
EMP Tools
Production ERP Newsletter …
Checkout
* API-based ** “good old” Shop2
UMP
Tracking
RankingSearch
MarketplaceShops*
Partner Area
DesignerDesignerDesigner
Shops** User Area
Website
Payments
Orders
CDN
Spreadshirt
2 Data Centers
26
Spreadshirt
2 Data Centers
27
~125ms latency*
* more or less but no guarantees
Spreadshirt
TCP: 2-way handshake
28
Spreadshirt
Browser load times
29
Spreadshirt
2 Data Centers + Content Delivery Network
30
> 1.000 Points of Presence
Spreadshirt
2 Data Centers + Content Delivery Network
31
> 1.000 Points of Presence
• Latency
• Bandwidth
• Performance
Spreadshirt
Performance Monitoring
32
> 1.000 Points of Presence
Spreadshirt
Managing data globally
33
Reminder: latency matters
•  Synchronous operations don’t work in practice over WAN
Spreadshirt
CAP Theorem
34
Consistency Availability
Partition Tolerance
Spreadshirt
CAP Theorem
35
Partition Tolerance
CP AP
Consistency Availability
Spreadshirt
Managing objects in the cloud
36
User generated content (vector & pixel images)
•  Some 10s of terabytes (TB) of data
•  2 typical sizes:
§  A few MB (originals)
§  A few KB (post-processed)
•  50.000 uploads per day at peak
•  Currently 42M+ objects
•  Read > write
Spreadshirt
Managing objects in the cloud
37
•  Traditional file systems & tools at their limit
§  Complicated structure with (too) many folders (shards) and files
§  Forget rsync & co.
Solution:
•  S3-API
§  A few “buckets”
§  Many “objects” per bucket
•  Ceph Object Storage as Backend
§  Strongly consistent (DC level)
•  Custom Geo-replication
§  Eventual consistent (globally)
Spreadshirt
Thank You
38
jns@spreadshirt.com

Spreadshirt Platform - An Architectural Overview