AppDynamics Business Transaction Guide
AppDynamics Business Transaction Guide
Introduction to Business
Transaction Discovery
APM221
Student Guide
Contents
Step 1: What is your APM Strategy? What do you want to accomplish with
AppDynamics?
Step 2: Instrument your application. Validate that the flow maps match your
expected architecture and design.
Step 3: Identify KEY Business Transactions and map them to your customer
experiences and functions. It is critical to understand and identify the most important
transactions that affect your customers’ experience.
Step 4: Set up SLA based Health Rules on key Business Transactions. Ensure you
are fully aware of the customer experience at all times. For each Health Rule,
identify actions to be taken. What do you do when you get a warning alert? A critical
alert? What are your workflows? This can and should include the ability to potentially
self-heal your applications if applicable.
Step 5: Show people. Create dashboards that reflect and measure the health and
performance of your application.
It’s far better to have a monitoring tool that can spot problems as they occur, then take
steps to capture relevant detail and even potentially perform some kind of remediation
activity to help ensure your customers aren’t impacted.
Do you know what your current Mean Time to Detect a problem is? How about the
Mean Time To Diagnose a problem? And how long do you need to fix problems?
It’s usually pretty hard to give clear answers on this, but with AppDynamics you get
measurements about what the problems are, when they occurred, along with the
detail to help you work out what is causing them.
Downtime is expensive, right? The quicker you can reduce your downtime, the less
money you will lose - both in terms of short-term revenue impacted as well as longer
term reputation damage.
War rooms are typically expensive and time-consuming, as you often have to gather
experts from various areas of your IT team to then figure out what has happened
and why. That all costs you money, time and hassle.
Finding out what goals different teams in your organization have should be considered a
driving force in how you approach setting up your AppDynamics monitoring solution.
The needs of Developers naturally differ from those of Helpdesk, as they do from those
of Operations, and the overall company goals must also be used to steer the
implementation and maximize the value you get from adopting AppDynamics.
Some specific questions you, or others in your company, may want answers to include:
Why am I losing customers?
Where are the bottlenecks in my system?
How can I figure out if the vendors I’m using are giving me poor service at certain
times?
Who’s killing my database with their thousands of SQL statements?
How much revenue do I lose when the Credit Card gateway breaks down?
Once App Agents are installed and configured, then you start to see Flow Maps in the
Controller. Getting agents into your systems is a key early step that has to happen
before monitoring can begin to work. Even once the App Agents are working, there is
going to need to be time spent creating Business Transactions and updating the BT
Rule Detection configurations so you see the specific BTs that are important for you.
Once Agents are installed and Flow Maps appear in the Controller, you should review
the Flow Map(s) for a ‘sanity check’ to confirm the mapped architecture makes sense
for your application(s) landscape.
Our goal is not to provide you with a Business Transaction for every single little action in
your system, or to somehow monitor every single icon on every webpage. We could do
that, and there are competitors who do as well, but it comes at a price - overhead. At
AppDynamics, we set out to run in Production with the lowest possible overhead to
avoid impacting how your applications function. We are built on the principle that you
don’t need to monitor absolutely everything, just the parts that matter. This drives
everything we do, including the limit on the number of BTs and the recommendation to
every customer that they identify which customer journeys matter most and don’t try to
measure every single one.
The idea is to monitor only the most important transactions - the ones that substantially
affect your customers and your business.
The more transactions you monitor, the more difficult it is to focus on the ones that
really matter.
Now that you have worked out what’s important to monitor, think about:
What is the normal level of performance you expect the system to deliver?
What is not acceptable in terms of response time, number of errors, number of slow
(or very slow) calls over time, etc.
Thinking in these terms, and identifying what your KPIs are, leads you to defining Health
Rules to enforce your expected performance levels.
Think about the Login BT. It might typically run at about 40ms with infrequent errors.
Now imagine that one night, because of a batch job impacting the database, the Login
BT changes to an average of 200ms response time. The customer isn’t going to notice
this as an issue, but you will want to be aware of it. Is it important enough to wake
someone up during the night, or would you be ok waiting until the next day to
investigate and resolve the issue?
The first step towards putting in place a fully proactive framework for your system is
identifying which BTs to develop Health Rules for.
The next step is to set up Policies and Actions to make AppDynamics do something
when a Health Rule is violated (e.g., send out an Email and/or SMS, do a Thread Dump
on a given JVM, restart a service using a script, post a HTTP request to an Admin
Console to increase scaling, or automatically generate a new ticket in your Jira system).
Login
A transaction such as Login should be very reliable and responsive. Depending on the
system, it might be an issue if it starts to take 500ms, but for many users that still
wouldn’t prompt them to immediately go elsewhere (i.e., your competitor), however if it’s
failing or taking several seconds, customers will likely start to get annoyed and begin
submitting support tickets or even not bothering to come back.
In the event that Login slows down, you might just want to be notified at a certain
performance threshold, and then actually wake someone up if it crosses a higher
threshold. Similarly, if the number of Login Errors rises above a certain point, you might
want to take immediate action such as waking someone up to figure out the problem.
Run a Report
Depends on whether or not this is an internal function used by employees, or something
that customers expect will be immediate or not: if perception is that there is a lot of data
to process to create the report, then several seconds duration before the response is
returned will be seen as ok.
Depending on the type of report and who the user is (employee or customer) you might
want to either just send a notification, or generate a ticket in Jira, or potentially message
someone to look into it right away.
Take payment
In this case it very much depends on what customer perception is of how long payment
takes. E.g., if they assume that their bank or other financial institution has to accept the
debit before your system confirms the payment has been successfully taken, then there
is potentially some leeway in how long is “too long.” You certainly do not want any
errors to occur during the transaction, leaving the customer unsure of whether the
payment went through or not.
If your payment gateway stops working, and your system is supposed to allow
purchases at any time of the day, then clearly immediate action is required, either in the
form of notifying the appropriate support staff immediately, or perhaps restarting a
service (if you know that this will resolve the problem).
Once you have identified and configured the right BTs and set up your Health Rules and
Actions, you’ll want to define Custom Dashboards so that you can see how your
systems are running at a glance, and others in your organization can also review how
particular systems are running, how many customers are using specific functionality,
how much revenue the business is earning, etc.
This diagram depicts how distributed applications communicate with one another. Entry
point and exit points are key transaction terms as they help you describe specific points
in the transaction flow. Note where Entry Points and Exit Points are placed.
Entry Points are the places in a process where execution of the code begins. In web
applications that have user interfaces these are often Servlet classes for Java
applications and ASP.NET classes for .NET applications. Other languages have similar
classes that identify Entry Points.
Exit Points are places in a process where execution of the code calls some external
service. The most common Exit Points are database calls and web service calls.
When the destination of an Exit Point is another service that is monitored by
AppDynamics then AppDynamics will employ the Tag and Follow mechanism
(discussed soon). For Tag and Follow to work, Exit Points from an upstream process
must match up to an Entry Point in a downstream process.
AppDynamics excels in providing extensive coverage for Entry Points and Exit Points in
modern applications.
The way that AppDynamics determines how downstream services are related is via a
concept called “Tag and Follow”. Tag and Follow means that AppDynamics decorates
outbound calls via an AppDynamics GUID (like for example from ECommerce-
Services to Order-Processing-Services and from ECommerce-Services to Inventory-
Services) to downstream tiers. The AppDynamics agents in the downstream systems
see that there is a GUID in the incoming call. This information is used to stitch together
a transaction topology.
Each AppDynamics agent has multiple communication channels for different purposes
that initiate connections to the Controller independently, and at different time intervals.
The agent configuration channel queries the Controller for any new configuration
changes, and downloads these changes when available, every 60 seconds.
The agent metric channel posts all new periodic metrics, including JMX, Windows
performance counters, and business transaction metrics to the Controller every 60
seconds.
If there are new business transactions that have not been seen before by the agent,
they are posted to the Controller for registration every 10 seconds.
The App Server Agent sends any new snapshots to the Controller every 20 seconds.
Certain events will also be sent to the Controller every 20 seconds. (Not all events
are generated at the App Server Agent level; the other events are created in the
Controller).
Automatic Discovery
When looking at a system that is going to be monitored using AppDynamics, beyond the
technical implementation phase of installing and configuring the App Agents, you have
to start by exploring the functionality.
You want to understand what the key transactions are, i.e., what are the most important
functions that customers do in the system? What might customers complain about?
If they were complaining that “It’s slow” or “it’s broken”, then “it” is the customer
experience of what they’re trying to do, which matters to them e.g., rent movie, rate
movie, etc.
These are going to be what you want your Business Transactions to monitor, to give
you the detailed insight into how things seem from the customer perspective.
Beyond those mentioned on the previous slide, AppDynamics offers a varying level of
support for other Java Frameworks and JVM Language Frameworks. In some cases,
auto detection is not enabled by default, or perhaps auto-naming isn’t available.
For .NET there is also extensive support for transaction detection for a range of
frameworks and Queuing technologies.
For a full list of supported frameworks, see the Java Supported Environments and .NET
Supported Environments pages of the AppDynamics Product documentation.
Even with a simple app like MovieZtream, examining the third segment in the URL
uncovers important information for some BTs.
The issue with seeing only two segment URLs is that they may not map strongly to the
customer experience. MovieZtream is relatively simple and this is reflected in the short-
ish URLs it has. Real-world systems are more complex and consequently have more
complex resource paths and request types.
In the case of MovieZtream, changing the number of segments to look at in the URL
from two to three, reveals additional transaction-specific information. The ‘customer’ BT
would include Rent, Rate, and Rents if limited to two URL segments. By changing it to
three segments we uncover addition BTs that are relevant to the customer experience.
You can update the number of URL segments to use in the Auto Discovery Rules (Java
Servlet or .NET ASP) under Configure Naming on the Rule Configuration tab.
A few minutes after the configuration change (switching to 3 segments), we should start
to see the new, more fine-grained BTs appearing. The old 2 segment BTs are not lost,
however, but gradually slip out of view as the Controller is typically only showing the last
X minutes of information. By default, the list of Business Transactions only shows ones
that have executed in the time frame selected. This can be changed under Filters.
Delete - AppDynamics may discover some Business Transactions that are not
significant to your business. You can remove those and just keep the transactions that
have higher business impact. Select any of the Business Transactions on the list, and
access More Actions > Delete Transactions.
Exclude - If you delete a transaction it may be re-discovered. If you don’t want that, use
Exclude Transaction instead.
Rename - You can rename Business Transactions to give them ‘human-friendly’ names
for easier reference. When you rename a BT you rename it for everyone.
Group - Grouping helps you access the consolidated key data, while keeping the
metrics for each Business Transaction still accessible. To create a group, you must
select more than one BT and then go to More Actions > Create Group.
We will look at the application of these operations as we begin to define Custom Match
Rules later in this course.
The Auto Discovery Rules are there when anyone installs AppDynamics - they are a
“one-size fits all” best estimation of what might be useful Business Transactions, but
never a good fit for any one specific customer.
First it’s important to understand the All Other Traffic buckets and how they work. Auto-
Discovery Rules may put many of your ‘valuable’ transactions in the All Other Traffic
buckets, and some unwanted BTs would show up in your BT list; both of which you
want to avoid.
AppDynamics will assign what would be independent BTs into the All Other Traffic
buckets, once it reaches the standard limits at either Application level (200) or Node
level (50).
In addition, if BT Lockdown is enabled, then any subsequent traffic (i.e., not being
captured by an existing BT) gets directed into the All Other Traffic bucket.
Notice that there is not just one bucket, but one per unique Tier where one or more BTs
have Entry Points.
The first step in hand-selecting BTs is turning on BT Lockdown. This freezes the current
set of BTs so unwanted BTs don’t continue to show up.
The next step in hand-selecting BTs is to switch from the default two-segment URI
detection setting to Full URI detection. This will give you a unique BT for every unique
URI detected.
While this may sound great (and you may think that is what you want), keep in mind the
general principle of aiming to have only a small number of BTs showing in your
Controller, which should reflect the key functions offered by your application. In other
words, we’re not done. We will continue to refine our BT selection in the next steps.
It is also worth mentioning the default limit of 200 BTs that a Controller account can
monitor (there is also a limit of 50 originating BTs on any 1 Node). Once the
AppDynamics detects traffic in excess of one of these limits, the surplus is dumped into
the All Other Traffic bucket.
We also recognize that your system may not be primarily driven by requests from your
end users’ web browsers, in fact it may be specifically focused on service requests -
providing it is built using a technology which we auto detect, then you will still see
unique BTs showing up for unique request paths coming into your application. Review
the other technologies listed in the Rule Configuration page for the .NET and Java Auto
Discovery rules to see the logic used to determine unique BTs.
Configuration of other detectable non-browser-based protocols/frameworks is less
granular and already set to ON by default.
After engaging BT Lockdown and configuring URI Naming to create unique BTs for
every unique URI, the next step in hand-selecting BTs is to delete all the current BTs so
all traffic will automatically go into the All Other Traffic bucket from this point forward.
NOTE: Be sure to turn all filters off before selecting BTs (including Performance Data
filter). Otherwise you will not delete filtered BTs.
Also available on this screen is the ability to exclude transactions. Excluding the
transaction disables the BT for metric processing purposes. When you delete a BT from
the list, the system removes its accumulated transaction metrics; when you exclude a
BT, the system retains its accumulated metrics along with its underlying configuration.
Once the Business Transactions are deleted, if the load on the monitored application
continues, then new traffic will end up in the All Other Traffic buckets (according to Tier
of Entry Point).
In the ‘All Other Traffic - mz_UI’ bucket, you see a familiar BT Flow Map. In the top right
corner there is a message informing you that this is a representation of all the traffic not
organized as individual Business Transactions, and a link to View Traffic Details.
The next step in hand-selecting BTs is to go into the All Other Traffic bucket and begin
to pick out the specific BTs you want and register them to upgrade them to first class
unique BTs.
Clicking on the View Traffic Details link displays a pop-up, with the actual unique
requests which COULD be upgraded to specific BTs.
Review the requests to determine which should be monitored with their own BT, and
which can potentially be grouped together for monitoring purposes.
Select a request to be monitored with its own unique BT and click the Register button.
The selected request becomes a specific BT in the Controller, and it will no longer show
up in the All Other Traffic bucket.
Remember, AppDynamics sets a limit of 200 unique BTs per account, and 50 per
originating Node, so in fact any time that your BT configuration allows more than those
limits, once the first 200 (or 50) BTs have been detected, anything over and above will
always end up in the All Other Traffic bucket.
While this results in this specific BT being displayed in the Controller for this
environment, it hasn’t created the Custom Match Rule needed for that the unique BT to
be found in case you ever want to delete all BTs again. Further, creating BTs via
Custom Match rules provides better control over how BTs are defined as well as an
easy mechanism to port rules from one controller to another.
Clicking the Register button creates a BT for the chosen request, with no extra work or
input. What does not happen at this point, however, is the creation of the logic
necessary to be able to discover the request again in future; should the BT ever be
deleted, or if you move AppDynamics configuration to a new environment. At this stage,
the BTs would have to be registered again by hand from the All Other Traffic bucket.
The missing piece to this puzzle is the creation by you of new Custom Match Rules
which provide the concrete logic for identifying a specific request and monitoring it from
then on with a particular BT. We will discuss creating Custom Match Rules in the next
topic.
Requests that show in in All Other Traffic buckets are not BTs.
Auto Discovery rules are a great way to potentially detect lots of traffic and get lots of
BTs without much configuration effort. However, they are relatively coarse-grained in
nature, meaning that they apply an ‘all or nothing’ logic to detecting and creating BTs,
which does not work perfectly for everyone.
More fine-grained control over detection of specific requests is available by using
Custom Match Rules (Include and Exclude Types).
Custom Match rules can have priorities assigned to them. This allows you to create
rules that create individual Business Transactions.
Higher priority numbers get evaluated first. The idea is that the “catch-all” rule will get
evaluated last and will catch all of the Business Transactions that do not match any of
the previous rules.
If deployed correctly this methodology will only create Business Transactions for which
there are Custom Match rules and will never create an “All Other Traffic – tiername”
category. This is because the “catch-all” rule will catch all of the overflow Business
Transactions and they will never fall into the “All Other Traffic - tiername” category.
If we want to ensure that there is always a BT for the MovieSearch functionality, we can
potentially use the full (unique) URL, or at least some part which uniquely distinguishes
it from other requests, to define a Custom Match Rule, as we will see in the coming
demonstration and slides.
On the Rule Configuration tab you have options based on the Agent Type and Entry
Point Type that you previously selected. If you chose Java and Servlet Entry Point
Type, then you are presented with options to allow you to focus on the detection of the
request based on some detail of the URI. Notice how you have a range of ways to
specify the Match Criteria for the URI.
Now all that remains to do is to highlight the new BT and Register it.
Returning to the BT List page, our new BT will now show up alongside any other
registered BTs and the All Other Traffic bucket within a few minutes.
The difference now is that even if we deleted the Search Movie BT, or migrated the BT
configuration to another environment, Search Movie would not need to be re-discovered
and re-registered.
If the BT configuration were migrated to another Controller Application, and BT
Lockdown was active in the new environment, then while the BTs with specific rules
would be discovered, they would still need to be registered once more.
Scopes can hold any combination of tiers within an application. A Default Scope,
including all tiers in the application, is included out of the box.
A scope can only contain Agents of the same language type.
One set of rules can be applied everywhere or rules can be customized per scope.
Rules can be modified and/or copied from one scope to another.
For more information, please copy and paste this URL into a separate tab in your
browser: https://learn.appdynamics.com/certifications
For information on specific certification tracks, please copy and paste the appropriate
URLs below: