Handle Complex
POST/PATCH requests in
     RESTful API
              Dmitry Petrov
      old.fightmaster@gmail.com
Product Fulfillment and Information Tracking




                         ProFIT


            The system of process
                  control printing


                                               Dmitry Petrov
Product Fulfillment and Information Tracking




                         ProFIT
    Daily:
    ~ 1 000 orders
    ~1 000 000 print productions
    1 hour of downtime ~ 25 000$

                                               Dmitry Petrov
RESTful API for ProFIT




               RESTful API

   ~ 60 entity
   ~100 API endpoints
   Complex business logic

                             Dmitry Petrov
RESTful API, examples GET

   GET /api/orders/12                             GET /api/orders/12/items/fg45sf54
   The server response:                           The server response:
   {                                              {
       "id": 12,                                      "id": "fg45sf54",
       "url": "http://localhost/api/orders/12",       "url": "http://localhost/api/orders/12/items/fg45sf54",
       "client": {                                    "product": "business cards",
           "firstname": "Dmitry",                     "quantity": 1000,
           "lastname": "Petrov",                      "previews": {
           "email": "",                                   "front": {
           "phone": null,                                      "large": "http://localhost/large/front.jpg",
           "address": {                                        "medium": "http://localhost/medium/front.jpg",
               "country": "Russia",                            "small": "http://localhost/small/front.jpg",
               "city": "Saratov",                         },
               "zip": 123456,                             "back": {
               "street": "Vavilova",                           "large": "http://localhost/large/back.jpg",
               "residentional": false                          "medium": "http://localhost/medium/back.jpg",
           }                                                   "small": "http://localhost/small/back.jpg",
       }                                                  }
   }                                                  }
                                                  }

                                                                                                          Dmitry Petrov
RESTful API, examples GET

   GET /api/machines/KARAT+1/hot-folders   GET /api/product-box-types/12-type/associations
   The server response:                    The server response:
   [                                       [
       {                                       {
            "path":"/home/somepath/",               "id": 1,
            "types": [                              "product": "business_cards",
                "34-f-Type",                        "quantity": 1000
                "33-S-Type",                   },
                ......                         ......
            ]                              ]
       },
       ......
   ]


   GET /api/press-sheets/134/label
   The server response:
   {
       "label": "epl string"
   }


                                                                                   Dmitry Petrov
RESTful API, examples POST / PUT

   POST http://localhost/api/orders,    POST http://localhost/api/press-sheets/12/transition
   PUT http://localhost/api/orders/12   Body of the request:
   Body of the request:                 {
   {                                        "transition": "start:printing:front",
       "id": 12,                            "note": null
       "client": {                      }
           "firstname": "Dmitry",
           "lastname": "Petrov",
           "email": "",
           "phone": null,
           "address": {
               "country": "Russia",
               "city": "Saratov",
               "zip": 123456,
               "street": "Vavilova",
               "residentional": false
           }
       }
   }


                                                                                    Dmitry Petrov
RESTful API, examples PATCH

   Object:                                         PATCH http://localhost/api/orders/12
   {                                               Body of the request:
       "id": 12,                                   {
       "client": {                                     "client": {
           "firstname": "Dmitry",                           "email": "",
           "lastname": "Petrov",                            "phone": null
           "email": "old.fightmaster@gmail.com",       }
           "phone": "8-888-999",                   }
           "address": {                            PATCH http://localhost/api/orders/12
               "country": "Russia",                Body of the request:
               "city": "Saratov",                  {
               "zip": 123456,                          "client": {
               "street": "Vavilova",                        "email": ""
               "residentional": false                  },
           }                                           "address": {
       }                                                    "street": "Vavilova",
   }                                                        "residentional": true
                                                       }
                                                   }


                                                                                          Dmitry Petrov
Become thoughtful . . .




                          Dmitry Petrov
Data Transfer Object




                           DTO
                                             DomainObject1

         DTO                                attribute1: String
                          Assembler
  attribute1: String
  attribute2: String   createDTO
                       updateDomainObject
  serialize
  deserialize
                                             DomainObject2

                                            attribute2: String




                                                                 Dmitry Petrov
Examples of DTO

   GET /api/orders/12
   The server response:
   {
       "id": 12,
       "url": "http://localhost/orders/12",
       "client": {
           "firstname": "Dmitry",
           "lastname": "Petrov",
           "email": "",
           "phone": null,
           "address": {
               "country": "Russia",
               "city": "Saratov",
               "zip": 123456,
               "street": "Vavilova",
               "residentional": false
           }
       }
   }


                                              Dmitry Petrov
Examples of DTO


 {
     "transition": "start:printing:front",
     "note": null
 }


 {
     "label": "epl string"
 }
 [
     {
          "path":"/home/somepath/",
          "types": [
              "34-f-Type",
              ......
          ]
     },
     ......
 ]


                                             Dmitry Petrov
The benefits of using DTO pattern




Reducing the number of queries
Independence from API
"Makes you think"




                                    Dmitry Petrov
Popular bundles and examples




   FOSRestBundle
   JMSSerializerBundle
   LiipHelloBundle
   FOSCommentBundle


                               Dmitry Petrov
JMSSerializerBundle & GET method


 GET /api/orders/12
 The server response:
 {
     "id": 12,
     "url": "http://localhost/api/orders/12",
     "client": {
         "firstname": "Dmitry",
         "lastname": "Petrov",
         "email": "",
         "phone": null,
         "address": {
             "country": "Russia",
             "city": "Saratov",
             "zip": 123456,
             "street": "Vavilova",
             "residentional": false
         }
     }
 }

                                                Dmitry Petrov
JMSSerializerBundle & POST method


POST /api/orders,
Body of the request:
{
    "id": 12,
    "client": {
        "firstname": "Dmitry",
        "lastname": "Petrov",
        "email": "",
        "phone": null,
        "address": {
            "country": "Russia",
            "city": "Saratov",
            "zip": 123456,
            "street": "Vavilova",
            "residentional": false
        }
    }
}


                                     Dmitry Petrov
JMSSerializerBundle & PATCH method




                                     Dmitry Petrov
JMSSerializerBundle




  $this->deserialize($request, 'RestOrderDTO', 'json');




                                                   Dmitry Petrov
JMSSerializerBundle




                  MERGE


   $this->merge($oldDTO, $newDTO);



                                Dmitry Petrov
Merging DTO




              Dmitry Petrov
JMSSerializerBundle & PATCH method




PATCH /api/orders/12
Request:
{
    "client": {
        "email": "",
        "phone": null
    }
}




                                     Dmitry Petrov
RESTful API, JMSSerializerBundle



 Problems / Disadvantages

GET - serialization of null values
PATCH - deserialized into an object
PATCH - merge null values
MERGE - a lot of useless code

                                     Dmitry Petrov
SimpleThingsFormSerializerBundle




 SimpleThingsFormSerializerBundle




                               July 15, 2012
                                         Dmitry Petrov
SimpleThingsFormSerializerBundle




                                   Dmitry Petrov
SimpleThingsFormSerializerBundle

GET /api/orders/12
The server response:
{
    "id": "12",
    "url": "http://localhost/orders/12",
    "client": {
        "firstname": "Dmitry",
        "lastname": "Petrov",
        "email": "",
        "phone": "",
        "address": {
            "country": "Russia",
            "city": "Saratov",
            "zip": "123456",
            "street": "Vavilova",
            "residentional": "false"
        }
    }
}


                                           Dmitry Petrov
SimpleThingsFormSerializerBundle




 Problems / Disadvantages

Converting data into string
Lack of support PATCH method (v. 2.0)
The ideological aversion
The dirty mix *Type and *DTO
                                        Dmitry Petrov
Отпуск




         Dmitry Petrov
Отпуск




         Dmitry Petrov
Отпуск




         Dmitry Petrov
Reinvent the wheel




             Requirements

   (Un)Serialization of objects
   Preservation of the type in data
   Metadata cache

                                  Dmitry Petrov
Reinvent the wheel




             Assumptions

   The output format is json
   Metadata is stored in yml
   There are get/set methods

                               Dmitry Petrov
Reinvent the wheel




            After 36 hours...   train Saratov - Kiev is comming 30 hours




   SimpleSerializer
   SimpleSerializerBundle


   Details can be found on the habr
                                                                  Dmitry Petrov
SimpleSerializer




                     Benefits

This is library
Separation of serialization rules and format
Absence voiced disadvantages
"Inteligent" deserialization


                                               Dmitry Petrov
SimpleSerializer & POST / PATCH




                                  Dmitry Petrov
SimpleSerializer & POST method




                                 Dmitry Petrov
RESTful API, validation




     What? Where? When?

   Parameters of requests
   Data transfer objects
   Business logic

                            Dmitry Petrov
RESTful API, validation




    Parameters of requests


   /api/orders/12
   /api/boxes/BOX-1-1
   /api/orders?valid=true
                            Dmitry Petrov
RESTful API, validation




      Routing requirements




                             Dmitry Petrov
RESTful API, validation




         ParameterChecker




                            Dmitry Petrov
RESTful API, validation




                          Dmitry Petrov
RESTful API, validation




                          Dmitry Petrov
RESTful API, validation


        AbstractRestController




                                 Dmitry Petrov
RESTful API, validation

                                                 PATCH /api/orders/12
 Object:
                                                 Body of the request:
 {
                                                 {
     "id": 12,
                                                     "client": {
     "client": {
                                                         "email": "",
         "firstname": "Dmitry",
                                                         "comment": "I'm hacker"
         "lastname": "Petrov",
                                                     }
         "email": "old.fightmaster@gmail.com",
                                                 }
         "phone": "8-888-999",
         "address": {
             "country": "Russia",
             "city": "Saratov",
             "zip": 123456,
             "street": "Vavilova",
             "residentional": false
         }
     }
 }




                                                                                   Dmitry Petrov
RESTful API, validation


 Object:                                     POST /api/press-sheets/12/transition
 {                                           Body of the request:
     "transition": "start:printing:front",   {
     "note": null                                "transition": "start:printing:front",
 }                                               "note": null,
                                                 "comment": "I'm hacker"
                                             }
                                             POST /api/press-sheets/12/transition
                                             Body of the request:
                                             {
                                                 "transition": "start:printing:front",
                                                 "comment": "I'm hacker"
                                             }




                                                                                         Dmitry Petrov
RESTful API, validation




      How, where and
      when to handle
      these situations?


                          Dmitry Petrov
REST APIs with Symfony2: The Right Way




                                         Dmitry Petrov
REST APIs with Symfony2: The Right Way




              Disadvantages


 Conventionalism
 Duplication of code
 Only works as filter
                                         Dmitry Petrov
SimpleSerializer




                "Inteligent"
               deserialization
       3 modes of deserialization:
   Strict, Medium strict, Non-strict
                         +
                   Support groups
                                     Dmitry Petrov
RESTful API, handle of DTO




                             Dmitry Petrov
RESTful API, handle of domain object




                                       Dmitry Petrov
RESTful API, testing




            Behat, PHPUnit
   Controllers


   Data access layer
   Service layer
                             Dmitry Petrov
RESTful API, Behat scenario




                              Dmitry Petrov
RESTful API, testing




                  Problems

   Run time:
   Behat ~ 90 minutes
   PHPUnit ~ 5 minutes

                             Dmitry Petrov
RESTful API, authentication




                      WSSE


Atom Authentication
How to create a custom Authentication Provider
EscapeWSSEAuthenticationBundle (v. 2.0)
MopaWSSEAuthenticationBundle (v. 2.1)

                                                 Dmitry Petrov
RESTful API, authentication




              WSSE Header

  X-WSSE: UsernameToken Username="bob",

  PasswordDigest="quR/EWLAV4xLf9Zqyw4pDmfV9OY=",

  Nonce="d36e316282959a9ed4c89851497a717f",

  Created="2003-12-15T14:43:07Z"




                                                   Dmitry Petrov
RESTful API, authentication




           Password digest


    Base64 (SHA1 (Nonce + CreationTimestamp + Password))




                                                      Dmitry Petrov
RESTful API, The End




                       Dmitry Petrov
RESTful API




          Any questions?
              @old_fightmaster




              https://github.com/opensoft
              https://github.com/fightmaster

                                        Special thanks to ProFIT team

                                                          Dmitry Petrov

Handle complex POST/PATCH requests in RESTful API

  • 1.
  • 2.
    Product Fulfillment andInformation Tracking ProFIT The system of process control printing Dmitry Petrov
  • 3.
    Product Fulfillment andInformation Tracking ProFIT Daily: ~ 1 000 orders ~1 000 000 print productions 1 hour of downtime ~ 25 000$ Dmitry Petrov
  • 4.
    RESTful API forProFIT RESTful API ~ 60 entity ~100 API endpoints Complex business logic Dmitry Petrov
  • 5.
    RESTful API, examplesGET GET /api/orders/12 GET /api/orders/12/items/fg45sf54 The server response: The server response: { { "id": 12, "id": "fg45sf54", "url": "http://localhost/api/orders/12", "url": "http://localhost/api/orders/12/items/fg45sf54", "client": { "product": "business cards", "firstname": "Dmitry", "quantity": 1000, "lastname": "Petrov", "previews": { "email": "", "front": { "phone": null, "large": "http://localhost/large/front.jpg", "address": { "medium": "http://localhost/medium/front.jpg", "country": "Russia", "small": "http://localhost/small/front.jpg", "city": "Saratov", }, "zip": 123456, "back": { "street": "Vavilova", "large": "http://localhost/large/back.jpg", "residentional": false "medium": "http://localhost/medium/back.jpg", } "small": "http://localhost/small/back.jpg", } } } } } Dmitry Petrov
  • 6.
    RESTful API, examplesGET GET /api/machines/KARAT+1/hot-folders GET /api/product-box-types/12-type/associations The server response: The server response: [ [ { { "path":"/home/somepath/", "id": 1, "types": [ "product": "business_cards", "34-f-Type", "quantity": 1000 "33-S-Type", }, ...... ...... ] ] }, ...... ] GET /api/press-sheets/134/label The server response: { "label": "epl string" } Dmitry Petrov
  • 7.
    RESTful API, examplesPOST / PUT POST http://localhost/api/orders, POST http://localhost/api/press-sheets/12/transition PUT http://localhost/api/orders/12 Body of the request: Body of the request: { { "transition": "start:printing:front", "id": 12, "note": null "client": { } "firstname": "Dmitry", "lastname": "Petrov", "email": "", "phone": null, "address": { "country": "Russia", "city": "Saratov", "zip": 123456, "street": "Vavilova", "residentional": false } } } Dmitry Petrov
  • 8.
    RESTful API, examplesPATCH Object: PATCH http://localhost/api/orders/12 { Body of the request: "id": 12, { "client": { "client": { "firstname": "Dmitry", "email": "", "lastname": "Petrov", "phone": null "email": "[email protected]", } "phone": "8-888-999", } "address": { PATCH http://localhost/api/orders/12 "country": "Russia", Body of the request: "city": "Saratov", { "zip": 123456, "client": { "street": "Vavilova", "email": "" "residentional": false }, } "address": { } "street": "Vavilova", } "residentional": true } } Dmitry Petrov
  • 9.
    Become thoughtful .. . Dmitry Petrov
  • 10.
    Data Transfer Object DTO DomainObject1 DTO attribute1: String Assembler attribute1: String attribute2: String createDTO updateDomainObject serialize deserialize DomainObject2 attribute2: String Dmitry Petrov
  • 11.
    Examples of DTO GET /api/orders/12 The server response: { "id": 12, "url": "http://localhost/orders/12", "client": { "firstname": "Dmitry", "lastname": "Petrov", "email": "", "phone": null, "address": { "country": "Russia", "city": "Saratov", "zip": 123456, "street": "Vavilova", "residentional": false } } } Dmitry Petrov
  • 12.
    Examples of DTO { "transition": "start:printing:front", "note": null } { "label": "epl string" } [ { "path":"/home/somepath/", "types": [ "34-f-Type", ...... ] }, ...... ] Dmitry Petrov
  • 13.
    The benefits ofusing DTO pattern Reducing the number of queries Independence from API "Makes you think" Dmitry Petrov
  • 14.
    Popular bundles andexamples FOSRestBundle JMSSerializerBundle LiipHelloBundle FOSCommentBundle Dmitry Petrov
  • 15.
    JMSSerializerBundle & GETmethod GET /api/orders/12 The server response: { "id": 12, "url": "http://localhost/api/orders/12", "client": { "firstname": "Dmitry", "lastname": "Petrov", "email": "", "phone": null, "address": { "country": "Russia", "city": "Saratov", "zip": 123456, "street": "Vavilova", "residentional": false } } } Dmitry Petrov
  • 16.
    JMSSerializerBundle & POSTmethod POST /api/orders, Body of the request: { "id": 12, "client": { "firstname": "Dmitry", "lastname": "Petrov", "email": "", "phone": null, "address": { "country": "Russia", "city": "Saratov", "zip": 123456, "street": "Vavilova", "residentional": false } } } Dmitry Petrov
  • 17.
    JMSSerializerBundle & PATCHmethod Dmitry Petrov
  • 18.
    JMSSerializerBundle $this->deserialize($request,'RestOrderDTO', 'json'); Dmitry Petrov
  • 19.
    JMSSerializerBundle MERGE $this->merge($oldDTO, $newDTO); Dmitry Petrov
  • 20.
    Merging DTO Dmitry Petrov
  • 21.
    JMSSerializerBundle & PATCHmethod PATCH /api/orders/12 Request: { "client": { "email": "", "phone": null } } Dmitry Petrov
  • 22.
    RESTful API, JMSSerializerBundle Problems / Disadvantages GET - serialization of null values PATCH - deserialized into an object PATCH - merge null values MERGE - a lot of useless code Dmitry Petrov
  • 23.
  • 24.
  • 25.
    SimpleThingsFormSerializerBundle GET /api/orders/12 The serverresponse: { "id": "12", "url": "http://localhost/orders/12", "client": { "firstname": "Dmitry", "lastname": "Petrov", "email": "", "phone": "", "address": { "country": "Russia", "city": "Saratov", "zip": "123456", "street": "Vavilova", "residentional": "false" } } } Dmitry Petrov
  • 26.
    SimpleThingsFormSerializerBundle Problems /Disadvantages Converting data into string Lack of support PATCH method (v. 2.0) The ideological aversion The dirty mix *Type and *DTO Dmitry Petrov
  • 27.
    Отпуск Dmitry Petrov
  • 28.
    Отпуск Dmitry Petrov
  • 29.
    Отпуск Dmitry Petrov
  • 30.
    Reinvent the wheel Requirements (Un)Serialization of objects Preservation of the type in data Metadata cache Dmitry Petrov
  • 31.
    Reinvent the wheel Assumptions The output format is json Metadata is stored in yml There are get/set methods Dmitry Petrov
  • 32.
    Reinvent the wheel After 36 hours... train Saratov - Kiev is comming 30 hours SimpleSerializer SimpleSerializerBundle Details can be found on the habr Dmitry Petrov
  • 33.
    SimpleSerializer Benefits This is library Separation of serialization rules and format Absence voiced disadvantages "Inteligent" deserialization Dmitry Petrov
  • 34.
    SimpleSerializer & POST/ PATCH Dmitry Petrov
  • 35.
    SimpleSerializer & POSTmethod Dmitry Petrov
  • 36.
    RESTful API, validation What? Where? When? Parameters of requests Data transfer objects Business logic Dmitry Petrov
  • 37.
    RESTful API, validation Parameters of requests /api/orders/12 /api/boxes/BOX-1-1 /api/orders?valid=true Dmitry Petrov
  • 38.
    RESTful API, validation Routing requirements Dmitry Petrov
  • 39.
    RESTful API, validation ParameterChecker Dmitry Petrov
  • 40.
  • 41.
  • 42.
    RESTful API, validation AbstractRestController Dmitry Petrov
  • 43.
    RESTful API, validation PATCH /api/orders/12 Object: Body of the request: { { "id": 12, "client": { "client": { "email": "", "firstname": "Dmitry", "comment": "I'm hacker" "lastname": "Petrov", } "email": "[email protected]", } "phone": "8-888-999", "address": { "country": "Russia", "city": "Saratov", "zip": 123456, "street": "Vavilova", "residentional": false } } } Dmitry Petrov
  • 44.
    RESTful API, validation Object: POST /api/press-sheets/12/transition { Body of the request: "transition": "start:printing:front", { "note": null "transition": "start:printing:front", } "note": null, "comment": "I'm hacker" } POST /api/press-sheets/12/transition Body of the request: { "transition": "start:printing:front", "comment": "I'm hacker" } Dmitry Petrov
  • 45.
    RESTful API, validation How, where and when to handle these situations? Dmitry Petrov
  • 46.
    REST APIs withSymfony2: The Right Way Dmitry Petrov
  • 47.
    REST APIs withSymfony2: The Right Way Disadvantages Conventionalism Duplication of code Only works as filter Dmitry Petrov
  • 48.
    SimpleSerializer "Inteligent" deserialization 3 modes of deserialization: Strict, Medium strict, Non-strict + Support groups Dmitry Petrov
  • 49.
    RESTful API, handleof DTO Dmitry Petrov
  • 50.
    RESTful API, handleof domain object Dmitry Petrov
  • 51.
    RESTful API, testing Behat, PHPUnit Controllers Data access layer Service layer Dmitry Petrov
  • 52.
    RESTful API, Behatscenario Dmitry Petrov
  • 53.
    RESTful API, testing Problems Run time: Behat ~ 90 minutes PHPUnit ~ 5 minutes Dmitry Petrov
  • 54.
    RESTful API, authentication WSSE Atom Authentication How to create a custom Authentication Provider EscapeWSSEAuthenticationBundle (v. 2.0) MopaWSSEAuthenticationBundle (v. 2.1) Dmitry Petrov
  • 55.
    RESTful API, authentication WSSE Header X-WSSE: UsernameToken Username="bob", PasswordDigest="quR/EWLAV4xLf9Zqyw4pDmfV9OY=", Nonce="d36e316282959a9ed4c89851497a717f", Created="2003-12-15T14:43:07Z" Dmitry Petrov
  • 56.
    RESTful API, authentication Password digest Base64 (SHA1 (Nonce + CreationTimestamp + Password)) Dmitry Petrov
  • 57.
    RESTful API, TheEnd Dmitry Petrov
  • 58.
    RESTful API Any questions? @old_fightmaster https://github.com/opensoft https://github.com/fightmaster Special thanks to ProFIT team Dmitry Petrov