Testing for fun in production Into The Box 2018
1 like327 views
This document discusses testing applications in production environments. It recommends putting unit tests in place and automating as much testing as possible. However, it also advocates for some manual testing in production with test data to catch real-world bugs and monitor application behavior and performance with actual users and loads. Guidelines are provided around using test data, credentials, and logging when testing in production environments.
![@markdrew
Feature('MainTest');
Scenario('test something', (I) => {
I.amOnPage('/');
I.fillField('#lst-ib', 'Into The Box');
I.click('#tsf > div.tsf-p > div.jsb > center > input[type="submit"]:nth-child(1)');
I.see('Into the Box is a yearly web technologies and software craftsmanship conference');
I.click('Into The Box 2018');
I.see('April 25-27, 2018 | Texas')
});
maintest_test.js](https://image.slidesharecdn.com/testingforfuninproduction-180507213923/85/Testing-for-fun-in-production-Into-The-Box-2018-40-320.jpg)
Testing for fun in production Into The Box 2018
- 1. TESTING IN PRODUCTION Fun* times getting your application tested
- 4. QUICK RECAP…
- 5. CMDdevelop • deploy • deliver @markdrew [email protected] http://cmdhq.io
- 6. WHAT WE ARE COVERING ➤ Background ➤ Workflow ➤ Unit Testing ➤ QA Testing ➤ Testing in Production ➤ Q&A (not QA)
- 7. BACKGROUND
- 8. BACKGROUND ➤ Complex Application ➤ Owned actual hardware at ISP ➤ Disaster Recovery (DR) ➤ WIDE range of platforms ➤ Range of usage patterns ➤ Lots of assets and data ➤ Issues with getting code to production ➤ 💩 code constipation ➤ 💥 Actual Disaster - ISP needs to move.
- 9. Web app Image Conv Meta indexing Image Conv Log
- 11. WORKFLOW
- 12. TIMETABLE Week1 Mon Tue Wed Thurs Fri Dev Dev Dev Dev Dev Dev QA Infra Week2 Mon Tue Wed Thurs Fri Dev Dev Dev Release Estimate Train/Est QA Test Test Test Infra Week3 Mon Tue Wed Thurs Fri Dev Dev Dev Dev Dev … QA Test Test Infra Prep Deploy
- 13. ENVIRONMENTS Development Unit Test QA Live
- 14. ENVIRONMENTS Development Unit Test QA Live
- 16. UNIT TESTING
- 18. TESTBOX ➤ Unit testing framework ➤ Not just for Coldbox apps! ➤ Also includes Mockbox ➤ Test individual logic of your code. ➤ Just had two sessions of it!
- 19. BASIC TEST component extends="testbox.system.BaseSpec" { function run() { describe( "Has a commentGateway", function() { beforeEach( function() { commentService = new core.objects.platform.comment.commentService(); entityId = 3078; } ); it( "should have a function called `getCommentGateway`", function() { makePublic(commentService, "getCommentGateway"); expect(commentService).toHaveKey("getCommentGateway"); } ); it( "should call `getCommentGateway` function and return a new instance", function() { makePublic(commentService, "getCommentGateway"); var commentGateway = commentService.getCommentGateway(); expect(commentGateway).toBeTypeOf("component"); } ); } ); }
- 20. MOCKBOX component extends="testbox.system.BaseSpec" { function run() { describe( "Can delete comments", function() { beforeEach( function() { commentService = new core.objects.platform.comment.commentService(); } ); it( "should return a error object", function() { var responseData = { "data": "Sorry, something went wrong.", "statusCode": "500", "statusText": "Internal server error" }; var commentService = prepareMock(new core.objects.platform.comment.commentService()); var commentGateway = prepareMock(new core.objects.platform.comment.commentGateway()); commentService.$("getCommentGateway", commentGateway); commentGateway.$("deleteComment"); var response = commentService.post(testData); expect(response).toBe(responseData) }); }); } }
- 21. TEST RESULTS
- 22. TEST INDEX
- 23. AUTOMATED TESTING (IN JENKINS) #!/usr/bin/env bash echo "Building the Unit Test image" container_name="unittests" image_name="unittests" docker build -t ${image_name} -f Dockerfile-UnitTests . mkdir -p reports echo "Starting..." docker run --rm --name ${container_name} -v "${PWD}/reports:/reports" ${image_name}
- 24. 😗 ❤
- 25. UNIT TESTING DOCKER IMAGE FROM ortussolutions/commandbox COPY . /app ENV IMAGE_TESTING_IN_PROGRESS true CMD cd $APP_DIR && $BUILD_DIR/run.sh && box testbox run runner=docker outputFile=/reports/ testresults.txt reporter=Text verbose=no && box server stop
- 26. ENVIRONMENTS Development Unit Test QA Live
- 27. QA TESTING
- 28. QA TESTING ➤ Manual testing ➤ Automated testing ➤ What cannot be automated?
- 29. MEET BOB…* * He’s a real person!
- 31. MANUAL TESTING
- 32. STANDARD PERFORMANCE AND FUNCTIONALITY TESTS
- 35. CodeceptJS
- 36. INSTALLATION ➤ NPM ➤ Node (obviously) ➤ Selenium Server ➤ CodeceptJS
- 37. @markdrew > npm install -g selenium-standalone > selenium-standalone install > selenium-standalone start Selenium
- 38. BUT
- 39. @markdrew { "tests": "./*_test.js", "timeout": 10000, "output": "./output", "helpers": { "WebDriverIO": { "url": "https://www.google.com", "browser": "chrome" } }, "include": { "I": "./steps_file.js" }, "bootstrap": false, "mocha": {}, "name": "codecept" } codecept.json
- 40. @markdrew Feature('MainTest'); Scenario('test something', (I) => { I.amOnPage('/'); I.fillField('#lst-ib', 'Into The Box'); I.click('#tsf > div.tsf-p > div.jsb > center > input[type="submit"]:nth-child(1)'); I.see('Into the Box is a yearly web technologies and software craftsmanship conference'); I.click('Into The Box 2018'); I.see('April 25-27, 2018 | Texas') }); maintest_test.js
- 42. AUTOMATED TESTING ➤ Pros ➤ Can automate a lot of the work ➤ Takes pictures of failures ➤ Nice logs and results ➤ Cons ➤ Doesn’t support everything ➤ File uploads ➤ File Downloads ➤ Drag and Drop events
- 43. ENVIRONMENTS Development Unit Test QA Live
- 45. TESTING IN PRODUCTION ➤Can be done. ➤Don’t do load testing though 😂 ➤Critical Paths
- 47. GUIDELINES ➤ Create your own test data. ➤ The naming convention of test data should be realistic. ➤ Do not play with other existing user’s data. ➤ Create your credentials to access the application. ➤ Never try load test on a production environment. ➤ Test only if there is less load on the application. ➤ Change emails/mail provider if you can ➤ Add to the logs / tagged output
- 48. BENEFITS ➤ Monitor behaviour realtime ➤ Run edge cases (network failure? Interrupted calls) ➤ API responses are … real ➤ More bugs are detected
- 50. Q&A (NOT QA)
- 51. IN SUMMARY ➤ Put unit tests in place ➤ Jon Clausen is a 🌟 ➤ Make space for QA Testing ➤ Automate what you can ➤ Manually test what you can ➤ Performance Check ➤ Live testing ➤ with your own data ➤ Against metrics ➤ Log “invisible” actions
- 52. CMDdevelop • deploy • deliver @markdrew [email protected] http://cmdhq.io