![compiled
languages
• A C++ RPC client
#include <iostream>
#include <string>
#include <boost/shared_ptr.hpp>
#include <thrift/transport/TSocket.h>
#include <thrift/protocol/TBinaryProtocol.h>
#include "gen-cpp/HelloSvc.h""
using namespace apache::thrift::transport;
using namespace apache::thrift::protocol;
using namespace boost;
int main() {
auto socket = make_shared<TSocket>("localhost", 8585);
socket->open();
auto protocol = make_shared<TBinaryProtocol>(socket);
HelloSvcClient client(protocol);
std::string msg;
client.hello_func(msg);
std::cout << "[Client] received: " << msg << std::endl;
}](https://image.slidesharecdn.com/thriftintro-140403175051-phpapp01/85/Apache-Thrift-a-brief-introduction-10-320.jpg)
![architecture
• User Code
– client code calls RPC
methods and/or
[de]serializes objects
– service handlers
implement RPC service
behavior
• Generated Code
– RPC stubs supply client
side proxies and server
side processors
– type serialization code
provides serialization for
IDL defined types
• Library Code
– servers host user
defined services,
managing connections
and concurrency
– protocols perform
serialization
– transports move bytes
from here to there](https://image.slidesharecdn.com/thriftintro-140403175051-phpapp01/85/Apache-Thrift-a-brief-introduction-17-320.jpg)
![resources
• Web
– thrift.apache.org
– github.com/apache/thrift
• Mail
– Users: user-subscribe@thrift.apache.org
– Developers: dev-subscribe@thrift.apache.org
• Chat
– #thrift
• Book
– Abernethy (2014), The Programmer’s Guide to
Apache Thrift, Manning Publications Co.
[http://www.manning.com/abernethy/]
Chapter
1 is free
Randy Abernethy
ra@apache.org](https://image.slidesharecdn.com/thriftintro-140403175051-phpapp01/85/Apache-Thrift-a-brief-introduction-19-320.jpg)
Apache Thrift, a brief introduction
- 1. -- Randy Abernethy, rx-m llc, 2014
- 2. polyglotism • Modern distributed applications are rarely composed of modules written in a single language • Weaving together innovations made in a range of languages is a core competency of successful enterprises • Cross language communications are a necessity, not a luxury
- 3. thrift • Apache Thrift – A high performance, scalable cross language serialization and RPC framework • What? – Full RPC Implementation - Apache Thrift supplies a complete RPC solution: clients, servers, everything but your business logic – Modularity - Apache Thrift supports plug-in serialization protocols: binary, compact, json, or build your own – Multiple End Points – Plug-in transports for network, disk and memory end points, making Thrift easy to integrate with other communications and storage solutions like AMQP messaging and HDFS – Performance - Apache Thrift is fast and efficient, solutions for minimal parsing overhead and minimal size – Reach - Apache Thrift supports a wide range of languages and platforms: Linux, OSX, Windows, Embedded Systems, Mobile, Browser, C++, Go, PHP, Erlang, Haskell, Ruby, Node.js, C#, Java, C, OCaml, ObjectiveC, D, Perl, Python, SmallTalk, … – Flexibility - Apache Thrift supports interface evolution, that is to say, CI/CD environments can roll new interface features incrementally without breaking existing infrastructure
- 4. • Service Interfaces are described with the Apache Thrift Interface Definition Language (IDL) • Client/Server Stubs are generated with the Apache Thrift IDL Compiler • The IDL Compiler can generate stubs in over 15 languages • Existing code modules are easily converted into RPC services using the Apache Thrift Server Library rpcservices
- 5. performance &reach • Thrift provides excellent performance for all but the most demanding solutions • Thrift provides broad reach, supporting a wide range of languages and platforms Custom Thrift REST Extreme Performance Extreme Reach High Performance Broad Reach Enterprise Compact Frameworks, C, etc. Web tech, scripting SOA, RPC Servers, Java, C#, C++, etc. WebEmbedded
- 6. thriftidl 1. Define the service interface in IDL 2. Compile the IDL to generate client/server stubs 3. Connect the server stubs to the desired implementation 4. Choose an Apache Thrift server to host your service 5. Call RPC functions like local function using the client stubs #sail_stats.thrift service SailStats { double GetSailorRating(1: string SailorName) double GetTeamRating(1: string TeamName) double GetBoatRating(1: i64 BoatSerialNumber) list<string> GetSailorsOnTeam(1: string TeamName) list<string> GetSailorsRatedBetween(1: double MinRating, 2: double MaxRating) string GetTeamCaptain(1: string TeamName) }
- 7. helloworld ~/thrift/hello $ ls -l -rw-r--r-- 1 dev dev 95 Mar 26 16:28 hello.thrift ~/thrift/hello $ thrift -gen py hello.thrift ~/thrift/hello $ ls -l drwxr-xr-x 3 dev dev 4096 Mar 26 16:31 gen-py -rw-r--r-- 1 dev dev 95 Mar 26 16:28 hello.thrift ~/thrift/hello $ ls -l gen-py drwxr-xr-x 2 dev dev 4096 Mar 26 16:31 hello -rw-r--r-- 1 dev dev 0 Mar 26 16:31 __init__.py ~/thrift/hello $ ls -l gen-py/hello -rw-r--r-- 1 dev dev 248 Mar 26 16:31 constants.py -rw-r--r-- 1 dev dev 5707 Mar 26 16:31 HelloSvc.py -rwxr-xr-x 1 dev dev 1896 Mar 26 16:31 HelloSvc-remote -rw-r--r-- 1 dev dev 46 Mar 26 16:31 __init__.py -rw-r--r-- 1 dev dev 398 Mar 26 16:31 ttypes.py ~/thrift/hello $ • Compiling IDL for an RPC service Generated Code • Interface Constants • HelloSvc Stubs • Sample Client • Package Init File • Interface Types
- 8. helloserver • A Python RPC Server
- 9. helloclient • A Python RPC Client
- 10. compiled languages • A C++ RPC client #include <iostream> #include <string> #include <boost/shared_ptr.hpp> #include <thrift/transport/TSocket.h> #include <thrift/protocol/TBinaryProtocol.h> #include "gen-cpp/HelloSvc.h"" using namespace apache::thrift::transport; using namespace apache::thrift::protocol; using namespace boost; int main() { auto socket = make_shared<TSocket>("localhost", 8585); socket->open(); auto protocol = make_shared<TBinaryProtocol>(socket); HelloSvcClient client(protocol); std::string msg; client.hello_func(msg); std::cout << "[Client] received: " << msg << std::endl; }
- 11. jvmlanguages • A Java RPC client operational
- 12. .netlanguages • A C# RPC client
- 13. • Apache Thrift uses a compiled IDL implementation – IDL is compiled generating stubs used at run time – Runtime serialization code allows for interface evolution • Apache Thrift IDL supports – Service Interface Definition – Type Definition • Service interfaces are exposed by Servers – Servers can implement many service interfaces – Interfaces can inherit from other interfaces • Types define serialization schemas – Types can be serialized to memory, disk or networks – Collections are supported (map, list, set) – Structures and Unions support composite types – DAGs are supported • New addition not widely implemented as of yet morethanrpc Interface Evolution Apache Thrift allows fields and parameters to be added and removed incrementally without breaking pre-existing code, allowing systems to grow incrementally over time (just like businesses) Particularly effective in dynamic CI/CD environments
- 14. abstractand isolated • Crafting an effective IDL requires understanding some of the most important things about your system – What are the key entities in your system and how are they described – What are their cardinalities – What are their keys – Which are immutable – Can you define idempotent interfaces for mutations – What are the operational affinity groups in your system – What is your system model and how will state and services be distributed across it • All of these things bear directly on IDL design • Free of implementation, you can get the concepts right and then choose the best languages and tools to implement them
- 15. entities&idl • Modern IDLs, like Apache Thrift, provide a rich set of tools for describing system entities (aka. messages) • Capturing and codifying the key system entities is prerequisite to effective interface specification • In some settings crafting the entities (e.g. an order) is all that the IDL need do Services are Optional!
- 16. commschemes • Streaming – Communications characterized by an ongoing flow of bytes from a server to one or more clients. – Example: An internet radio broadcast where the client receives bytes over time transmitted by the server in an ongoing sequence of small packets. • Messaging – Message passing involves one way asynchronous, often queued, communications, producing loosely coupled systems. – Example: Sending an email message where you may get a response or you may not, and if you do get a response you don’t know exactly when you will get it. • RPC – Remote Procedure Call systems allow function calls to be made between processes on different computers. – Example: An iPhone app calling a service on the Internet which returns the weather forecast. Apache Thrift is an efficient cross platform serialization solution for streaming interfaces Apache Thrift provides a complete RPC framework
- 17. architecture • User Code – client code calls RPC methods and/or [de]serializes objects – service handlers implement RPC service behavior • Generated Code – RPC stubs supply client side proxies and server side processors – type serialization code provides serialization for IDL defined types • Library Code – servers host user defined services, managing connections and concurrency – protocols perform serialization – transports move bytes from here to there
- 18. • The Thrift framework was originally developed at Facebook and released as open source in 2007. The project became an Apache Software Foundation incubator project in 2008, after which four early versions were released. • 0.2.0 released 2009-12-12 • 0.3.0 released 2010-08-05 • 0.4.0 released 2010-08-23 • 0.5.0 released 2010-10-07 • In 2010 the project was moved to Apache top level status where several additional versions have been released. • 0.6.0 released 2011-02-08 • 0.6.1 released 2011-04-25 • 0.7.0 released 2011-08-13 • 0.8.0 released 2011-11-29 • 0.9.0 released 2012-10-15 • 0.9.1 released 2013-07-16 • 0.9.2 released 2014-06-01 • 1.0.0 released 2015-01-01 versions it is difficult to make predictions, particularly about the future. -- Mark Twain, Yogi Berra, etc. Open Source Community Developed Apache License Version 2.0
- 19. resources • Web – thrift.apache.org – github.com/apache/thrift • Mail – Users: [email protected] – Developers: [email protected] • Chat – #thrift • Book – Abernethy (2014), The Programmer’s Guide to Apache Thrift, Manning Publications Co. [http://www.manning.com/abernethy/] Chapter 1 is free Randy Abernethy [email protected]