publishing, Sitecore

Publishing strategies


Publishing can be quite an issue for content authors and make an impact on performance. However if you use the right strategy, there really isn’t any issues.

One of the issues with publishing can be performance. When you publish one or more items the HTML cache is cleared, which means that if you have some performance heavy presentations, it will increase the loading time of the page, until the cache has been rebuild. Further if you use the staging module and it is configured to it, all the data caches are cleared as well. This can cause the site to take up to 30 seconds to reload your pages – especially if they are data heavy. You can read more about Sitecore Caching here and read about the issues with cache and the staging module here.

Note that Sitecore recently released a staging module which supports partial item caching, but the HTML cache is still cleared. You can read more about the new module on Alex Shybas blog.

I don’t know if performance impact when publishing is specific to Sitecore or is generally a problem in most CMS’, but I reckon it is the later. However Sitecore has a special problem with regards to caching. As Sitecore isn’t page based as many other CMS’ (this is also one of its biggest advantages), it makes it difficult to have a partial cache clearing – especially with regards to HTML cache. A page can be constructed by several content items, and there are no immediate binding between a page and the items, which it is constructed of, so all HTML caches needs to be cleared even though you just publish one item.

Another issue is the usability for content authors. Some of our clients were complaining about being queued all the time when they initiated a publish and it could take a very long time before they got their content published. What often happened was that when they initiated a publish, the modal dialog reported, that they were queued. Instead of waiting for it, they closed the window and continued editing. A few minutes later they would initiate another publish, but was once again told that they were queued. When this process repeated itself for the 50 active content authors, you can imagine the queue could get quite big.

Another thing we stumbled upon was, that some content authors used publishing instead of preview. Every time they wanted to view their work in progress to check if it looked alright frontend-wise, they published the item and viewed the page as if it was already completed and ready for actual publish.
I know we are not the only Sitecore partner experiencing this, as I have talked to other implementation partners and received questions about it on Learn Sitecore.

So why do I suggest it isn’t a problem? Well… we aren’t really experience any issues with our clients or with performance issues caused by publishing anymore. This is a result of more than one thing, but the primary solution was to use scheduled publish.

I don’t know why, but using this publishing strategy this can be a really big issue for clients. We often heard remarks like: “We need to publish content immediately”… But common! I really haven’t worked with many organizations, where this is an actual issue. What content do these organizations have, which is so important, that they can’t wait 15 minutes (at tops) to be published? I have consulted clients with huge sites, which is absolutely critical to the business and the funny part is that these companies rarely have a need to have immediate publishing. They are normally used to having a longer waiting period for the rest of their web services due to replications tasks between 100s of servers and rebuilding of caching. So why do the smaller companies have a bigger need for immediate publishing? The short answer – they don’t!

So what do we do to convince our client to use scheduled publish? We do more than one thing: First of all we explain to them, what impact publishing has. Then we allow a single or a few admins to have publish rights, so that they can have immediate publishing. Further we set the “do not publish” mark on all newly created items, so that they won’t be accidently published while they’re not finished. The authors will have to unclick the setting, when they think the item is ready for publishing.

Finally we have developed something, which in my opinion is rather cool! We acknowledged that people respond best to waiting time, if they have an indication of when the waiting will come to an end. This fact is also one of the reasons why they have timers on traffic lights in Copenhagen:

Traffic light with timer

7 seconds to green light 🙂

We decided to use the same strategy for scheduled publishing. If the content authors would know, when the next scheduled publish is due, they would be much more satisfied with the idea of not having immediate publishing. So we – in Pentia – developed a timer which is shown in the task bar in Sitecore: 

scheduled publish timer

The scheduled publish timer in action

The “Trinvis udgivelse” is Danish and mean incremental publish. So this timer tells the content author, that there is 12 minutes and 48 seconds to the next incremental publish. The functionality also supports multiple timers, so on some solutions we also use it to indicate, when an import from an external database is due and so forth. Cool, right?

These things combined we really don’t have any issues, with convincing our clients to use scheduled publishing.

Standard

16 thoughts on “Publishing strategies

  1. Steve Green says:

    That timer is an AWESOME idea.

    I’ve been in many a meeting over this exact same issue and like you said it’s really an uphill struggle. Visual feedback like this though would likely be enough to convince them otherwise.

  2. Pingback: Tweets that mention Publishing strategies « Molten Core -- Topsy.com

  3. Hi Jens

    Love your article, a very good strategy for publishing.

    Does it have a problem when you make a publish, but no content items has been added for publishing? im thinking will this clear the cache aswell? or is all you have to do, a check to see if any items is in the publishing que, and cancel the publish if no items is there?

  4. Hi Jens,

    Good to hear that I’m not the only one with the same experience ;-).
    It’s absolutely true that publishing requires a lot of resources. With the latest versions of Sitecore, the system becomes more and more intelligent on cache clearings, but still it’s always an expensive action. A famous computer expert ever told me: There’s no such a thing as optimalization, there’s a thing as making the computer do less.
    Guess you’re well describing this approach.

    Kind regards,

    Alex de Groot

  5. Hi Jens,

    What a GREAT Article, – and a recommendable solution with the timer.

    You mention that HTML cache is a serious issue. In fact, this is not really an issue… It is rather Sitecores item cache which causes these challenges.

    Allow me to explain:
    One of the challenges with (stand alone servers or multiple delivery servers) has traditionally been that the publishing server, upon the end of publishing, executes a full cache clearing, which means HTML (web controls) are removed from cache as well as the item cache.

    As HTML cache is cleared every single web control on the requested page is re-rendered. While a web control such as a menu only output the contents of, say 30 items, – in order to render the menu, Sitecore must first read all templates and field types, data from the core database, re-initiate security etc. In addition, a menu often iterates many more items that it outputs in order to complete its menu logic (for example if it should only display items of a specific template type).

    This means, the first web control being read triggers that the delivery server must fill the cache with tens of thousands of items. Next control being rendered will utilize large parts of the item cache (basic template, site initialization and some items now resides in memory) and will mean less load on the server as it only need to read not yet cached items from the data storage. Third web control will apply even less load on the server, etc.

    If Sitecore could solve the challenge with Item cache, clearing HTML cache should not (hopefully) affect the server. We at Sitecore regard the Item cache clearing (and refilling) as the most CPU consuming process of these). Fortunately, partial item cache clearing is 100% solved with the Sitecore Twin Peaks edition (The soon-to-be-released Sitecore version for scalability).

    Of course, this does not render your technique obsolete. In fact, this would work great in conjunction with Twin Peaks.

  6. Jens Mikkelsen says:

    Thanks for the comements everyone!

    @Mikkel Madsen: To my knowledge the cache is cleared even though there is no new items. We haven’t experienced a problem with it though, as it only publishes every lets say 15 minutes.

    @Jerrong: As it has been developed in and by Pentia, I’m not sure if my boss is interested in sharing it for free, but I’ll ask him though.

    @Lars Nielsen: I agree with you. It shouldn’t be a problem clearing the HTML cache. If that is an issue you probably have something wrong in your code. But I have to often seen in solutions I’ve debugged that an RSS feed or a news list iterates through all content items in the content tree, meaning that if you clear the HTML cache it will have quite an impact.
    Looking forward to Twin Peaks

  7. Could you give more specifics on your scheduled publish technique? It’s not clear how this is different than user-initiated publishes except for the schedule. So suppose you have a scheduled publish every 15 minutes? Wouldn’t that mean 96 (24*4) significant site delays each day?

  8. Jens Mikkelsen says:

    Hi Aren,

    First of all it doesn’t solve the issue with the staging module clearing the item cache. For that take a look at the new module with partial item cache.

    4 publishes per hour is in my opinion no issue. The HTML cache shouldn’t be so significant for your site. I have experienced clients with performance issues and when I took a look at the log they had above 100 publishes per hour in prime time.
    The publishing process in it self requires some ressources from the server, to that comes rebuilding of cache and the actual work of serving the visitors. All that load simply slows down the site especially if you have some data heavy presentations. If you stick with 4 publishes per hour, you shouldn’t be able to see a real impact on performance.

  9. Thanks! At my institution, any kind of traditional publish means a 30 – 60 second rendering delay, and that happening 4x/hr wouldn’t be acceptable. I also confirmed with Sitecore staff that the newer Staging module is what we need. We’re currently on a modified setup with the Stager, but I’m looking forward to returning to Staging because I also need the file publishes.

  10. Ivan says:

    Great idea as usual! I agree that awareness of publishing due time is very important for content authors and helps minimize number of manual publishing options.

  11. Eldblom says:

    @Aren: I would embrace this opportunity to optimise your code and setup. Fact is, Sitecore is fast – very fast – when running on the right environment, and with the right custom code. The problem is though, that very small issues in the hardware setup(poorly set up SQL server, not enough memory, too much load) and in the code (reading too many items, bad integration or architecture) can have a huge impact on the performance of your Sitecore solution.
    Your rule of thumb should be: always have your solution running smooth without HTML caching, after a publish, and with no warnings in the log (do not ignore the thresholds). If you rely on HTML caching, or have long startup times after publish, your website stability will suffer in times of high load.

  12. Eldblom said: “Your rule of thumb should be: always have your solution running smooth without HTML caching, after a publish, and with no warnings in the log (do not ignore the thresholds). If you rely on HTML caching, or have long startup times after publish, your website stability will suffer in times of high load.”

    Yikes. I have a lot of rendering/code optimization to do! 🙂

  13. Pingback: InfoDesktop « Coffee => Coder => Code

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s