Wasmで広がるEnvoyとIstioの世界

Transcript

1. Takeshi Yoneda, Software Engineer, Tetrate.io Kubernetes Meetup Tokyo #40 WasmͰ޿͕ΔEnvoyͱIstioͷੈք

2. • Takeshi Yoneda (Ϛελέ) / Twitter, Github: @mathetake • Software

   Engineer at Tetrate.io • “Paid” OSS dev: Envoy, Istio, Proxy-Wasm, TinyGo • C++ committer of Proxy-Wasm project • Creator of Go SDK for Proxy-Wasm • Contributor of V8 whoami

3. Software Design 2021೥3݄߸ʹࠓ೔ͷ࿩୊ʹ͍ͭͯͷهࣄΛدߘ͠·ͨ͠ʂ

4. 1. Introduction to WebAssembly 2. Background: Envoy’s extensibility 3. Proxy-Wasm:

   WebAssembly For Proxies 4. Proxy-Wasm in Istio 5. Challenges and Future Agenda

5. 1. Introduction to WebAssembly

6. Wasm = CNCF’s tech to watch in 2021

7. • Stack-basedͳԾ૝Ϛγϯͱͦͷ࢓༷ • ݩʑ͸ϒϥ΢β(JS)ͷߴ଎Խ͕໨త • asm.js -> WebAssembly(Wasm)΁ͱਐԽ • ࢓༷ΛಡΊ͹෼͔Δ͕Ұݴ΋

   “host” ΁ͷཁٻ͕ͳ͍ • Portable, Platform-agnostic, Open-ended • Run at near-native speed: ΊͬͪΌ଎͍(※࣮૷ʹΑΔ) • Security: ελοΫ͕ϓϩάϥϜ͔Βݟ͑ͳ͍ͱ͔ͦ͏͍͏ͷ WebAssembly 101

8. • ༷ʑͳݴޠ͔ΒίϯύΠϧՄೳ: C, C++, Rust, Go(TinyGo), AssemblyScript • ౰ॳ͸js΁ͷ૊ΈࠐΈ͕લఏ: ͦΕͧΕͷݴޠ͕ಠࣗͷ

   “glue.js”Λ࣋ͭ • ίϯύΠϥڞ௨ͷ“Platform”λʔήοτ͕ͳ͍(͍΍, jsͳΜ͚ͩͲ͞, Έ͍ͨͳ) • VMͱͯ͠༏ल&ίϯύΠϥج൫΋͋Δͷʹ໪ମͳ͍ • ϒϥ΢βͷ֎Ͱ΋࢖͍͍ͨ WebAssembly 101

9. • Wasm <-> Host OSͷ࿩͠ํ(ΠϯλʔϑΣΠε)=SystemcallΛඪ४Խ͠·͠ΐ͏ • Wasm΁ͷίϯύΠϥͷ“platform”λʔήοτʹ͠Α͏ • WASI (WebAssembly

   System Interface)ͷొ৔ WebAssembly 101 https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webassembly-system-interface/

10. • WASIΛ࣮૷ͨ͠ϥϯλΠϜ͕ొ৔ • WAVM, Wasmtime, Wasmer, Lucet, ౳ʑ • “ී௨ͷϓϩηε”ͱಉ͡Α͏ʹWasm͕ಈ͘Α͏ʹͳΔ

    • KrustletͳͲͷ໺৺తϓϩδΣΫτͷొ৔ Wasm gets out of web browsers

11. • WASI = Wasmͷ࢓༷ʹ͋Δ “host function” Λsystem call޲͚ʹඪ४Խͨ͠΋ͷ • ݁ہABI

    (Wasm <-> Hostͷ࿩͠ํ)ܾ͑͞ΊΕ͹ͳΜͰ΋Ͱ͖Δ • ೚ҙͷΞϓϦͷதͰVMΛಈָ͔͍ͯ͜͠͠ͱ͕Ͱ͖ͦ͏ʂ • WASIʹݶΒ༷ͣʑͳABI͕ొ৔ • Blockchain༻ͷABI • k8s֦ு༻ͷABI • Proxyαʔό༻ͷABI(Proxy-Wasm) Wasm gets out of web browsers WASI ABI Linux / Darwin / Windows / … Wasm Virtual Machine

12. • WASI = Wasmͷ࢓༷ʹ͋Δ “host function” Λsystem call޲͚ʹඪ४Խͨ͠΋ͷ • ݁ہABI

    (Wasm <-> Hostͷ࿩͠ํ)ܾ͑͞ΊΕ͹ͳΜͰ΋Ͱ͖Δ • ೚ҙͷΞϓϦͷதͰVMΛಈָ͔͍ͯ͜͠͠ͱ͕Ͱ͖ͦ͏ʂ • WASIʹݶΒ༷ͣʑͳABI͕ొ৔ • Blockchain༻ͷABI • k8s֦ு༻ͷABI • Proxyαʔό༻ͷABI(Proxy-Wasm) Wasm gets out of web browsers Proxy-Wasm ABI Envoy / Nginx / ATS / … Wasm Virtual Machine

13. 2. Background: Envoy’s extensibility

14. What is Envoy?

15. • “Cloud-native high-performance edge/middle/service proxy” • CNCF Graduated Project, Github

    Star: 16,000+ • Written in C++ • αʔϏεϝογϡͷData planeͱͯ͠།Ұແೋͷଘࡏ What is Envoy?

16. • ϓϩΩγαʔόʔͳͷͰuse case͕ແݶ • Envoyʹ͸౰વϓϥάΠϯ(֦ு)ػߏ͕ଘࡏ • ֦ுͷྫ • ࣾ಺ಠࣗͷAuthn/Authz ػೳΛೖΕ͍ͨ

    • ಛघͳϓϩτίϧΛѻ͍͍ͨ • MySQL, Redis, DynamoDB, etc. Envoy’s extensibility

17. Envoy’s extensibility

18. • Envoy͸੩తϦϯΫ͕લఏ • C++Ͱॻ͔ͳ͍ͱ͍͚ͳ͍ • มߋ൓өͷͨΊʹશ୆࠶ىಈ͕ඞཁ • StableͳABI͕ܾ·͍ͬͯͳ͍ • ֦ுػೳΛ௥Ճ͢Δʹ͸

    1. Upstream΁Ϛʔδ͢Δ —> ಛघ(private)ͳϢʔεέʔε͸ෆՄ 2. ಠࣗϏϧυΛ͢Δ —> upstream΁ͷ௥ै͕େม(಺෦ͷAPI͕unstable) Envoy’s extensibility

19. • ղܾ͍ͨ͠՝୊͸ҎԼͷ௨Γ • ಈతʹϓϥάΠϯͷload͕Ͱ͖Δ • ෳ਺ݴޠͷαϙʔτ • EnvoyࣗମͷಠࣗϏϧυ͸ඞཁͳ͍ • Security-awareͳ࢓༷

    Envoy’s extensibility

20. • ͜ΕΒͷཁ๬͸IstioͷνʔϜ͔Β্͕͍ͬͯͨ • ~v1.4ͷMixerγεςϜͷෛ࠴ • latency΍Istioࣗମͷෳࡶੑ • MixerࣗମΛEnvoyʹຒΊࠐΈ͍ͨ • ॊೈͳ֦ுػߏ͕Envoyʹཉ͍͠

    Istio Mixer: past

21. • ղܾ͍ͨ͠՝୊͸ҎԼͷ௨Γ • ಈతʹϓϥάΠϯͷload͕Ͱ͖Δ • ෳ਺ݴޠͷαϙʔτ • EnvoyࣗମͷಠࣗϏϧυ͸ඞཁͳ͍ • Security-awareͳ࢓༷

    Envoy’s extensibility

22. • ղܾ͍ͨ͠՝୊͸ҎԼͷ௨Γ • ಈతʹϓϥάΠϯͷload͕Ͱ͖Δ • ෳ਺ݴޠͷαϙʔτ • EnvoyࣗମͷಠࣗϏϧυ͸ඞཁͳ͍ • Security-awareͳ࢓༷

    Envoy’s extensibility

23. • ղܾ͍ͨ͠՝୊͸ҎԼͷ௨Γ • ಈతʹϓϥάΠϯͷload͕Ͱ͖Δ • ෳ਺ݴޠͷαϙʔτ • EnvoyࣗମͷಠࣗϏϧυ͸ඞཁͳ͍ • Security-awareͳ࢓༷

    Envoy’s extensibility

24. • ղܾ͍ͨ͠՝୊͸ҎԼͷ௨Γ • ಈతʹϓϥάΠϯͷload͕Ͱ͖Δ • ෳ਺ݴޠͷαϙʔτ • EnvoyࣗମͷಠࣗϏϧυ͸ඞཁͳ͍ • Security-awareͳ࢓༷

    Envoy’s extensibility

25. • ղܾ͍ͨ͠՝୊͸ҎԼͷ௨Γ • ಈతʹϓϥάΠϯͷload͕Ͱ͖Δ • ෳ਺ݴޠͷαϙʔτ • EnvoyࣗମͷಠࣗϏϧυ͸ඞཁͳ͍ • Security-awareͳ࢓༷

    Envoy’s extensibility
26. None

27. • ϓϥάΠϯ = WasmͷόΠφϦͱͯ͠Envoyʹ഑෍ • Envoyͷ“தͰ” WasmͷVMΛಈ͔͢ • Envoy <->

    Wasmͷ࿩͠ํ(ABI)Λstableʹ͠Α͏ • ͢΂ͯͷ՝୊ΛΫϦΞ • ೚ҙͷݴޠ, ಈతload, ηΩϡΞ, stableͳABI Extending Envoy with Wasm ಠࣗͷABI

28. • ͦ΋ͦ΋ϓϩΩγαʔόͷ֦ு༻ABI͸ීวతͳ΋ͷͳ͸ͣ • ྫ: Http Header/Body/Trailerʹରͯ͠◦◦͢Δ, tcpίωΫγϣϯʹରͯ͠xx͢Δ • Envoy͔Β੾Γ཭ͤΔͷͰ͸ʁ =>

    Proxy-Wasmͱ͍͏Envoy͔Βಠཱͨ͠ϓϩδΣΫτʹ Extending Envoy with Wasm Proxy-Wasm

29. 3. Proxy-Wasm: WebAssembly For Proxies

30. • https://github.com/proxy-wasm • ϓϩΩγαʔόͷ֦ுػߏͷͨΊͷABIͱSDK౳ͷ։ൃͷͨΊͷϓϩδΣΫτ • ։ൃ͸࣮࣭తʹ͸EnvoyίϛϡχςΟ(ݱঢ়GoogleͱTetrate) • LinkerdͳͲ΄͔ͷϓϩΩγք۾΋ࢀೖͷؾ഑ Proxy-Wasm: WebAssembly

    for Proxies

31. • ̐ͭͷݴޠͷSDK͕͋Γ, ؆୯ʹcompatibleͳWasm΁ͷίϯύΠϧ͕Մೳ • C++, Rust, Go, AssemblyScript • Hostͷެ࣮ࣜ૷͸C++ͷΈ

    • https://github.com/proxy-wasm/proxy-wasm-cpp-host • VMͱͯ͠ Wasmtime, V8, WAVM͕ݱঢ়ར༻Մೳ • Envoy, Apache Traffic Server(PoC)౳͸͜ͷެ࣮ࣜ૷ΛϥΠϒϥϦͱͯ͠࢖͏ Proxy-Wasm: WebAssembly for Proxies

32. • Spec: https://github.com/proxy-wasm/spec • WasmͷϓϩάϥϜͱϓϩΩγαʔόͷABI(࿩͠ํ)ΛఆΊͨ΋ͷ Proxy-Wasm Spec

33. Proxy-Wasm Spec: Example 1. ϩΪϯάͷͨΊʹ “Wasm͔Β”ݺͿؔ਺

34. Proxy-Wasm Implementation: Example 1. proxy_logͷ࣮૷ in ϗετ https://github.com/proxy-wasm/proxy-wasm-cpp-host/blob/master/src/exports.cc#L854-L864

35. Proxy-Wasm Implementation: Example 1. proxy_logͷ࣮૷Λhostʹظ଴ https://github.com/proxy-wasm/proxy-wasm-rust-sdk/blob/master/src/hostcalls.rs#L20-L32

36. Proxy-Wasm Spec: Example 2. ίωΫγϣϯཱ֬࣌, ϗετ͔Βݺ͹ΕΔ “Wasm಺”ͷؔ਺ Tcp data͕౸ண࣌, ϗετ͔Βݺ͹ΕΔ

    “Wasm಺”ͷؔ਺

37. Proxy-Wasm Spec: Example 2. http request header౸ண࣌, ϗετ͔Βݺ͹ΕΔ “Wasm಺”ͷؔ਺ http

    request body͕౸ண࣌, ϗετ͔Βݺ͹ΕΔ “Wasm಺”ͷؔ਺

38. Proxy-Wasm Implementation: Example 2. SDK಺Ͱ࣮૷ & export https://github.com/tetratelabs/proxy-wasm-go-sdk/blob/main/proxywasm/abi_l7.go#L21-L40

39. Proxy-Wasm Implementation: Example 2. Envoy಺ͷEventϧʔϓ https://github.com/envoyproxy/envoy/blob/master/source/extensions/common/wasm/context.cc#L1577-L1587 https://github.com/proxy-wasm/proxy-wasm-cpp-host/blob/master/src/context.cc#L308-L319

40. Proxy-Wasm Implementation: Example 2. Envoy಺ͷEventϧʔϓ https://github.com/envoyproxy/envoy/blob/master/source/extensions/common/wasm/context.cc#L1577-L1587 https://github.com/proxy-wasm/proxy-wasm-cpp-host/blob/master/src/context.cc#L308-L319 proxy-wasm-cpp-host಺ͷ onResponseHeadersΛݺͿ

41. Proxy-Wasm Implementation: Example 2. Envoy಺ͷEventϧʔϓ https://github.com/envoyproxy/envoy/blob/master/source/extensions/common/wasm/context.cc#L1577-L1587 https://github.com/proxy-wasm/proxy-wasm-cpp-host/blob/master/src/context.cc#L308-L319 proxy-wasm-cpp-host಺ͷ onResponseHeadersΛݺͿ “Wasm಺”ͷproxy_on_request_headers

42. Proxy-Wasm in Envoy: strace-ish log WasmͷVM<->Envoy HTTP filterͷձ࿩ϩά

43. Proxy-Wasm in Envoy: strace-ish log WasmͷVM<->Envoy Network filterͷձ࿩ϩά

44. • 1 VM / (Plugin, Worker Thread) • Nativeͷ֦ுͷதͰproxy-wasm- cpp-hostΛ࢖༻

    • 1೥൒΋ͷؒ “envoy-wasm”ͱ͍͏ forkઌͰ։ൃ • 2020/10ʹupstream΁merge • v1.17ͰॳͷstableϦϦʔε Proxy-Wasm in Envoy: Impl. model

45. Proxy-Wasm in Envoy: released in v1.17

46. 4. Proxy-Wasm In Istio

47. • Proxy-WasmʹΑΓMixer͕Envoy಺ʹ૊ Έࠐ·Εͨ • γϯϓϧ͔ͭLatencyͷݮগ • ϚΠΫϩαʔϏεΛࢧ͑ΔIstioࣗ਎͕Ϟ ϊϦεʹͳͬͨ࿩୊ʹ Istio v1.5~

    Before After

48. • github.com/istio/proxy ʹProxy-Wasm SDKΛ༻͍ͨIstioಠ֦ࣗுͷ࣮૷ Istio: official plugins in Proxy-Wasm

49. • چTelemetryػೳ͸Proxy-WasmϓϥάΠϯͱͯ͠࠶࣮૷͞Εͨ (stats plugin) Istio: official plugins in Proxy-Wasm https://github.com/istio/proxy/blob/master/extensions/stats/plugin.h

50. • ࣮͸͜ΕΒͷϓϥάΠϯ͸ਖ਼֬ʹ͸σϑΥϧτͰ͸“Wasm”Ͱಈ͍͍ͯͳ͍ • Proxy-WasmͷABIΛ༻͍ͯ͸͍Δ͕NativeʹCompile & Envoyʹstatic link • NullVmͱզʑ͕ݺΜͰ͍Δ࢓૊Έ •

    EnvoyͷWasm Extension͕·ͩalphaεςʔλεͳͨΊ • ҰํͰɺWasmͱͯ͠΋ίϯύΠϧՄೳͳΑ͏ʹίʔυ͕ॻ͔Ε͍ͯΔ • ͍ۙকདྷ(v1.10~)σϑΥϧτͰWasm VM಺ͰϓϥάΠϯ͕ಈ͘Α͏ʹͳΔ͸ͣ Istio: official plugins in Proxy-Wasm

51. • ݱঢ়Wasm plugin༻ͷ API(CRD)͸ଘࡏ͠ͳ͍ • EnvoyFilterͰؤுΔ͔͠ͳ͍ How to deploy plugins

    in Istio: API

52. • ݱঢ়Wasm plugin༻ͷ API(CRD)͸ଘࡏ͠ͳ͍ • EnvoyFilterͰؤுΔ͔͠ͳ͍ • Wasm-SIG಺Ͱ1st classͳ APIΛࡦఆத

    How to deploy plugins in Istio: API

53. • ݱঢ়Istio (Istio-agent)͸, http(s)Ͱserving͞ΕͨWasmόΠφϦͷΈLoadՄ • OCI-imageͱͯ͠WasmͷbinaryΛOCI registryʹ֨ೲ͢ΔྲྀΕ͕͋Δ • https://github.com/engineerd/wasm-to-oci •

    https://github.com/solo-io/wasm-image-spec • Proxy-Wasm༻ͷOCI-image specΛIstio Wasm-SIG಺Ͱࡦఆத: ΄΅ݻ·ͬͨ • ͍ۙকདྷOCI registryʹpush͞ΕͨWasmΛIstio͕αϙʔτ͢ΔΑ͏ʹͳΔ • (࣮૷͸๻ͷTODOͰ͢…) How to deploy plugins in Istio: Delivery

54. Future of Workflow of Istio Wasm plugins Build Push OCI-registries

    k apply -f wasm.yaml Image: hoge.com/my-plugin:v1.10 config: …

55. 5. Challenges and Future

56. • ೚ҙͷϓϩάϥϜ͔ΒͲ͏ͷΑ͏ʹϗετΛकΔ͔? • Proxyαʔό͸ϛογϣϯΫϦςΟΧϧͳιϑτ΢ΣΞ • ςετ͸ॻ͍ͯ΋ॻ͍ͯ΋ॻ͖͖Εͳ͍ • Wasmࣗମ͕ηΩϡΞͱ͸͍͑ɺಛఆͷύεͰΫϥογϡ͢Δ͜ͱ΋͋ΓಘΔ Challenges in

    Proxy-Wasm

57. • ύϑΥʔϚϯεͷ໰୊ • GC෇͖ͷݴޠ͸Proxy-Wasm޲͚ͷGCΞϧΰϦζϜΛ։ൃ͠ͳ͍ͱ͍͚ ͳ͍? • Nativeʹൺ΂͓ͯΑͦ50%஗͍ͱ͍͏ϕϯνϚʔΫ • EnvoyશମͷॲཧൺͰ͸ແࢹͰ͖Δఔ౓ͳͷͰ͸? ͱ͍͏࿩΋͋Δ

    Challenges in Proxy-Wasm

58. • RuntimeͲΕબ΂͹ྑ͍ͷ͔໰୊ • ࣮ߦ଎౓΍ϝϞϦ࢖༻ྔ౳ͷτϨʔυΦϑ Challenges in Proxy-Wasm

59. • αϙʔτݴޠ, ͦ͜·Ͱଟ͘ͳ͍Α͏ͳ….? • WasmͷΤίγεςϜࣗମ͕·ͩະख़ Challenges in Proxy-Wasm

60. • ABI͕Envoyͷ࣮૷ʹ͍ͩͿد͍ͬͯΔ(౰ͨΓલͱ͍͑͹౰ͨΓલ…) • 1VM͕ෳ਺ͷϦΫΤετΛࡹ͘ͱ͍͏ઃܭ • V8 / WAVMͷoverhead͕എܠ • 1VM

    = 1 requestͱ͍͏Ϟσϧͷ৔߹ʹෆཁͳઃఆ͕͍͔ͭ͋͘Δ Challenges in Proxy-Wasm

61. • ςετͱσόά͕͔ͳΓਏ͍ • ςετͷͨΊʹEnvoyΛಈ͔͢…? • Go SDKͰ͸EnvoyͷΤϛϡϨʔλΛGoͰॻ͍ͯ, nativeίϯύΠϧ Ͱςετ: ݶք͕͋Δ

    • WasmଆʹdebugͷͨΊͷ࢓༷͕ͳ͍ͷͰ, ֤ݴޠ͝ͱʹ৭ʑ • LLVMϕʔεͷݴޠͳΒDWARF͕όΠφϦʹೖͬͯΔͷͰ৭ʑͰ͖ Δ͕Proxy-Wasm HostଆͰະ࣮૷(Ջ͕͋ͬͨΒ΍Δ) • Stack trace͕औΓͮΒ͍(Ϣʔβʔۭ͔ؒΒ͸stack͸ݟΕͳ͍ͷͰ) Challenges in Proxy-Wasm

62. • WASIͷ஥ؒೖΓ…? • ࣮ࡍProxy-Wasm͸͋ΔछͷWASIͷ֦ுͰ͋Δ • Pre-Proposal phaseͱͯ͠WASIͷProposalʹ͍Δ • https://github.com/WebAssembly/WASI/blob/master/docs/ Proposals.md

    Future of Proxy-Wasm

63. • Deployํ๏ͷσϑΝΫτελϯμʔυͷཱ֬ • IstioͰͷ࣮૷͸͜Ε͔Β • υΩϡϝϯςʔγϣϯ΍developer toolingपΓͷ֦ॆ Future of Proxy-Wasm

64. • ·ͩ·ͩEarly days (Join us!): server-side Wasmͷ੒ޭྫͱͳΓ͍ͨ Future of Proxy-Wasm

65. Future of Cloud Native Wasm 2015 2019/03 2020/03 WASIͷొ৔ WASMͷొ৔

    Proxy-Wasmͷొ৔ 2013/03 asm.jsͷొ৔ 2020/04 Krustletͷొ৔ 202x/yy ????

66. • Wasm͸ηΩϡΞͰportableͳόΠφϦϑΥʔϚοτ+Ծ૝Ϛγϯͷ࢓༷ • CNCF͕༧ଌ͢ΔΑ͏ʹ, ༷ʑͳ৔໘Ͱར༻͕૿͖͑ͯͦ͏ • Cloud Native Wasm DayͷτʔΫΛݟΔͱಈ޲͕ݟ͑ͯ͘Δ?

    • Proxy-WasmʹΑΓEnvoy/Istio͕WasmͰ֦ுͰ͖ΔΑ͏ͳੈք͕طʹདྷ͍ͯΔ • Isito͚ͩͰͳ͘, େاۀ͕طʹproductionͰಈ͔͍ͯ͠Δ • ·ͩ·ͩൃల్্Ͱ͕͢, 2021೥தʹ͸͔ͳΓ੒ख़͢ΔݟࠐΈ ·ͱΊ

67. We are hiring! https://www.tetrate.io/careers/ Work Anytime and Anywhere + Unlimited

    paid time off