Creating a custom Akamai replication agent in AEM

Adobe Experience Manager | AEM/CQ | Apache Sling

Creating a custom Akamai replication agent in AEM

Replication is central to the AEM experience. AEM is about content and replication is how you move that content across servers. You're most likely using at least two of the out-of-the-box replication agents provided by Adobe: your default agent activates content from author to publish and your dispatcher flush agent clears your Dispatcher cache. AEM provides several other replication agents for tasks such as replicating in reverse (publish to author), moving content within the Adobe Marketing Cloud products such as Scene 7 and Test and Target, and static agents for replicating to the file system. This blog post details the steps used to create your own custom replication agents.

Akamai replication agent selection dialog

The following sample project demonstrates a custom replication agent which purges Akamai CDN (Content Delivery Network) cached content. The full code for this blog is hosted on GitHub. There are three pieces to this project: the transport handler, the content builder, and the replication agent's user interface.

Transport Handler

TransportHandler implementations control the communication with the destination server and determines when to report back a positive ReplicationResult to complete the activation and when to report back a negative ReplicationResult returning the activation to the queue.

The default HTTP transport handler sends an authenticated request to the destination server specified in the "Transport URI" setting and expects an HTTP response with a status code of "200 OK" in order to mark the activation as successful. The Akamai CCU REST API responds with a status code of "201 Created" for a successful purge request. Therefore, a custom Transport Handler was utilized and given the responsibility for sending the POST requests, looking for 201 responses and returning the proper ReplicationResult.

The transport handler service determines which transport handler to use based on the overridden canHandle method. You'll notice that any replication agent configured with a "Transport URI" that begins with http:// or https:// will be handled by AEM's HTTP transport handler. The convention is to create a unique URL protocol/scheme and have your transport handler's canHandle method watch for Transport URIs that start with your URL scheme. For example, by navigating to a clean instance's Agents on Author page, you'll find default AEM replication agents using http://, static://, tnt://, s7delivery:// and repo://. In this example, the transport handler is activated on the amakai:// scheme and uses the hard coded Akamai API REST endpoint. A popular convention is to set your transport handler's Transport URIs that start with something like "foo-", which allows the user to configure their replication agent with either "foo-http://" or "foo-https://" and have your transport handler simply remove the custom prefix before making the HTTP request.

Akamai replication agent's Transport URI settings

The transport handler also handles other ReplicationActionTypes. The Akamai example implements the standard "Test Connection" feature of replication agents by making a GET request to the Akamai API endpoint which expects a "200 OK" response in return - it's simply testing the replication agent's configured username and password. The example does not implement other replication action types such as deletions, deactivations, and polling (reverse).

When you speak of a replication agent, you're first thought will probably be of the default agent that moves content from author to publish via HTTP. Likewise, I've been discussing the Akamai transport handler example and its usage of HTTP GET and POST requests. However, it's important to note that your transport handler doesn't need to make HTTP calls. For example, you can write an FTP transport handler or a transport handler that interacts with the server's file system like the static replication agent does. Your transport handler can do anything as long as it returns a positive or negative replication result to update the queue.

Content Builder

ContentBuilder implementations build the body of the replication request. Implementations of the ContentBuilder interface end up as serialization options in the replication agent configuration dialog along side the Default, Dispatcher Flush, Binary less, and Static Content Builder options.

The provided Akamai example project could have been done without implementing a content builder; the logic in the content builder could have been completed in the transport handler as the transport handler created it's own request anyways. Another thing to consider is whether you need the session as ContentBuilder implementations gives you that while TransportHandler implementations do not.

A perfect example of utilizing the content builder is in Andrew Khoury's Dispatcher refetching flush agent where the default HTTP transport handler is still used to communicate with the Dispatcher and only the HTTP request body needed to be built out in order for Dispatcher to fetch and re-cache content.

User Interface

By implementing a content builder, the user can simply use a default replication agent and choose the custom serialization type. However, the Akamai replication agent requires the following custom configurations:

  • an option to remove versus invalidate content
  • an option to purge resources versus purging via CP Codes
  • an option to purge the production versus staging domain
  • the reverse replication option removed

Akamai replication agent dialog set to purge CP codes

A clean user interface was provided in order for users to implement and configure the Akamai replication agent. To accomplish this, a custom cq:Template as well as a corresponding cq:Component including the view and dialog was made. The easiest way is to copy the default replication agent from /libs/cq/replication/templates/agent and /libs/cq/replication/components/agent to /apps/your-project/replication and update the agent like any other AEM component.

To keep things clean and simple, the Akamai replication agent component inherits from the default replication agent by setting the sling:resourceSuperType to cq/replication/components/agent. The only update needed to the copied component was the dialog options and the agent.jsp file as it contains JavaScript to open the dialog for which you need to update the path. Any additions to the dialog can be retrieved through the TransportContext's getConfig().getProperties() ValueMap.


Ankur Chauhan | January 06, 2016 at 03:58 AM | Reply

Hi Nate, Thanks for this post.

Isaías Sosa | January 08, 2016 at 03:44 AM | Reply

Thank you for sharing, a lot of work on this post.

Imad ElAli | January 08, 2016 at 08:40 AM | Reply

Hi Nate, Excellent post indeed. Have you considered the case of multiple publish instances and the need to delay the flush until all dispatchers have been flushed ? Any thoughts on that? Thanks

Nate Yolles | January 21, 2016 at 05:21 PM | Reply

Hello Imad. These are certainly valid questions that need to be considered for your unique project. On my current project, we have custom workflows that trigger replication agents on the author instance for our Dispatchers and for Akamai. Since this method of activation isn't a part of our normal release path, the agents are set to "Ignore default" on the Agent Settings "Triggers" tab, and they're initiated programatically. We're not currently delaying the Akamai purge in any way and we haven't experienced any issues as of yet. You also want to think about your strategy holistically; do you need to purge Akamai on demand or could you rely on your Akamai TTL (time to live) settings?

Luo | January 22, 2016 at 08:36 PM | Reply

Looks like a clean and better implementation than even listener. When I activate a page there is often a list of DAM assets that need to go with the page. It looks to me like each asset is sent as a single request to Akamai. I thought it would be better if we queue up the requests and send as a batch to Akamai. Any thoughts?

Luo | January 22, 2016 at 08:40 PM | Reply

One more thing here: The template/component targets AEM6.1. A Sightly upgrade of the template and the component would be a lot cleaner.

Nate Yolles | February 25, 2016 at 02:59 AM | Reply

You could write your custom replication agent with Sightly if you choose. However, as of this writing replication in AEM has not been updated to the Touch UI; I only needed to overlay a single file in order to change a single line of JS, so I felt the work to convert to Sightly would be premature at this time.

EvanAEM | January 29, 2016 at 06:31 AM | Reply

Thanks Nate, So how ContentBuilder fits into your context. I assume you are not using it but putting it here as an addition to TransportHandler.

Nate Yolles | February 25, 2016 at 02:53 AM | Reply

In this example the ContentBuilder is being used to access the activated resources and add any vanity URLs to the purge/invalidation list. It's not necessarily needed as the behavior could take place in the TransportHandler if your desired. However, the ContentBuilder provides access to the session.

EvanAEM | January 29, 2016 at 08:17 AM | Reply

Also what would be trigger for this agent - manual or automatic? Automatic is what most use cases need.

Nate Yolles | February 25, 2016 at 02:50 AM | Reply

A custom replication agent can be triggered in the same manner as an other replication agent. Check/uncheck "Ignore default" in the agent settings and/or trigger the agent programmatically as usual.

Saravanan Dharmaraj | March 24, 2016 at 05:34 PM | Reply

Excellent post ...! Thanks

Ankur Mittal | March 30, 2016 at 05:37 PM | Reply

Hi Nate. Your replication agent has been very helpful for me so far. I am trying to fix a scenario where agent gets triggered with any sort of node deletions/activations/deactivations. e.g. When some node from etc is programmatically deleted from author, agent gets queued up with the deactivate request for same path, which should not happen. Any suggestions how that can be handled?

Nate Yolles | July 12, 2016 at 10:44 PM | Reply

Without knowing the full details of your application, one possible solution is to set your Akamai Replication Agent trigger to "Ignore default" and utilize a Replication Event Listener to programmatically replicate with the Akamai agent on activation of only certain nodes.

Mit | June 20, 2016 at 02:51 PM | Reply

Hi Nate, I tried using this custom agent. I'm getting below error : Error while building replication content java.lang.IllegalArgumentException: Could not find configuration for domain 'production' Any idea on what have I missed?

Vaibhav | July 06, 2016 at 12:57 AM | Reply

Did you got the solution for this issue? I am also getting the same issue not sure what I missed. Error : Error while building replication content java.lang.IllegalArgumentException: Could not find configuration for domain 'production'

Nate Yolles | July 12, 2016 at 10:34 PM | Reply

Please read lines 103-110 of the ContentBuilder implementation. In this demonstration I'm using the Externalizer service to create the full URL for Akamai to purge based off the resource path. You can replace that section of code with your own method, probably some string manipulation to concatenate the domain with the path, or you can configure the Externalizer service at http://localhost:4502/system/console/configMgr/ with the "production" domain.

Josh | July 26, 2016 at 09:40 AM | Reply

Hi Nate, I got to know from Akamai that "REST CCU API will be deprecated soon - you need to update the library to use the new OPEN framework. " Will this agent work with OPEN as it is or some changes will require in OPEN framework?

Mit | August 03, 2016 at 08:59 AM | Reply

Hi Nate, While using this agent, the test connection gives 200 OK in response. But when I activate some resource, it shows blocked in queue with following logs: HttpResponseProxy{HTTP/1.1 403 Forbidden [Date: Wed, 03 Aug 2016 08:55:04 GMT, Server: Apache, Vary: Authorization, Content-Length: 192, Allow: GET, HEAD, POST, Keep-Alive: timeout=15, max=100, Connection: Keep-Alive, Content-Type: application/api-problem+json]} Can you please help me with this?

David | September 08, 2016 at 04:41 PM | Reply

Hey Nate, excellent work, thanks. I've seen you listed AEM 6.1 as a requirement. Do you see any reason this shouldn't work on 6.0, as we're tied to that in a current project of ours. I just tried installing it, but got a Replication triggered, but no agent found! when testing the connection. However, I am unsure if this is because of a msiconfiguration or incomaptibilities with 6.0 Thanks, David

Frank | September 30, 2016 at 02:12 AM | Reply

Hi Nate. This is great work! Your replication agent has been extremely helpful in helping my team develop a replication agent to replicate content metadata to a semantic database. The only issue we have come across is that the ContentBuilder's log messages sometimes do not appear in the replication log. Have you encountered this behavior before?

Upasana | June 29, 2017 at 03:18 PM | Reply

Great work Nate. This document has been very helpful.

Tushar | July 05, 2017 at 10:15 AM | Reply

Great Work Nate, but I wanted if you have tested the new fast purge feature of Akamai V3. If you could write something up to change the implementation for Akamai V3 using fast purge it will be a great help.

Leave a Comment