Magebase

Free Magento Extension – Customer Comments In The Checkout

Robert Popovic — Mon, 30 Apr 2012 04:00:47 +0000

Most e-commerce sites provide a way for customers to add comments to the order when they are checking out. Surprisingly, Magento does not provide built in functionality to deal with this simple requirement. This means that we will need to add this functionality as a custom extension and there are several extensions on the Magento extensions directory that provide this functionality.

So why are we adding yet another customer comment extension to the mix? We feel that our solution achieves the task with minimal and unobtrusive code and is in line with the Magento order management workflow.

Let’s see in a few screenshots how this extension shows up once it is installed.

1. Extension Configuration

The extension has a configuration section under: System > Configuration > Sales > Checkout > Magebase Checkout Comment Options:

The only configuration parameter is Enabled/Disabled

All it controls is whether the extension functionality is enabled or disabled. No rocket science there.

2. Checkout – Shipping Step

The extension comes with a layout file under app/design/frontend/base/default/layout/magebase/checkoutcomment.xml and a couple of phtml files under app/design/frontend/base/default/template/magebase/checkoutcomment/. Out of the box, the extension will automatically display the Customer Comment field on the shipping methods step in the one page checkout. No action should be necessary to make this field show unless you have a heavily customized checkout or you are using a custom checkout extension.

The customer comment field on the shipping step

Note that the extension doesn’t supply its own CSS so you will need to style appropriately.

3. Order Administration

The customer comment appears in the Comment History

Once the customer has completed the checkout and placed the order, we need to know where the customer comment information will be shown on the order management page. This is where the extension is ‘clever’ and keeps in line with the Magento order workflow. The extension will simply add the customer comment to the Comments History as shown in the screenshot to the right. This means that the extension doesn’t alter the default database to add custom columns to any Magento entities but uses the built in functionality to its advantage.

From an order workflow perspective, this makes sense as any notes and comments around the order are kept in the same area.

4. Order Confirmation Email

Another serendipitous consequence of using the built in Comments History functionality is that the customer comment is automatically sent out in the order confirmation email without any extra effort or additional template changes.

The customer comment in the confirmation email

How it works

If you lift the ‘hood’ off this extension, you will see that the code is very simple. It leverages off built in Magento features to provide this simple functionality in an unobtrusive way by using an event observer, the comments history method and the output="toHtml" layout attribute to force itself to render on the checkout template. The phtml is structured based on the Magento customer widget field implementation.

Suffice to say, the single line of code (in Observer.php) that makes it all happen is:

$observer->getEvent()->getQuote()->getShippingAddress()->setCustomerNotes($orderComment)->save();

Download

Magebase_CheckoutComment

Summary

Hopefully, we have supplied the community with a simple, free and useful extension and shown how to utilize built in Magento features to minimize our own programming effort and create efficient, performance conscious code.

Note: This extension was developed and tested on Magento 1.6.2.0 but should work fine for Magento CE 1.4.x and up, PE, EE 1.8.0 and up. There is no warranty or guarantee for the functionality and I don’t provide support for this extension. Use at your own risk and ALWAYS backup your database before installing.

Understanding the Grid Serializer Block

Paryank Kansara — Mon, 16 Apr 2012 05:00:56 +0000

Limitations of multi-select boxes

There are many cases in an ecommerce platform where some entities are related to each other with one-to-many relationship. Familiar examples are Upsell, Cross-sell and Related Products. One product may have many upsell, cross-sell or related products. Let’s take Upsell Products for example. Each product can have more than one Upsell product. To accept data for managing and storing this kind of relationship through a form or admin interface, multi-select boxes are normally sufficient, but this approach has a few limitations:

It can be used to display only one property of the entity. In our example, we can display product names in a multi-select box, but we can’t easily display other attributes like SKU, price etc.
The user can only select multiple entities but cannot supply additional data for each selection. In our example, the form can accept a selection of products but cannot accept data for the position of each product.
When there is huge number of options, multi-select boxes can become bulky and difficult to manage.
It cannot provide sorting or filtering so that user can easily pick choices from a subset of results.

The Grid Component as a Form Field

To overcome the limitations of a simple multi-select box, we should have an advanced element that has features like:

Ability to select/unselect multiple entities
Ability to display more information about entities
Ability to accept additional user input data
Should support pagination when there is a large number of entities
Should support sorting and filtering by multiple parameters
Should be unified so that it can be used for different entities and different data

Fortunately, Magento provides some very interesting components, which are very useful in the development of extensions involving the admin interface. These components are so powerful that developers have to write very little to almost ‘no’ template code to create an intuitive user interface.

One of these components is the ‘Grid’. It provides built-in pagination, sorting, and filtering. The Grid is widely used in the admin interface for listing entities like products, orders, invoices, customers etc. But my most favorite use of the Grid is as a form element. Magento provides an excellent solution to overcome the above described limitations of multi-select boxes by using the grid as a form element for accepting data for creating one-to-many relationships.

$this->addColumn('in_products', array(
    'header_css_class' => 'a-center',
    'type'            => 'checkbox',
    'name'            => 'in_products',
    'values'          => $this->_getSelectedProducts(),
    'align'           => 'center',
    'index'           => 'entity_id'
 ));

When a Grid is used as a form element as a replacement for multi-select boxes, normally, the first column type is defined as ‘checkbox’. So each row in the grid displays a checkbox. Users can tick the checkboxes to select particular rows. This is more intuitive than selecting multiple options in a multi-select box.

Data preservation problem between AJAX reloads

At first glance, this mechanism seems perfect. But, the grid also has pagination, sorting and filtering. Each time these operations are performed, the grid is reloaded through an AJAX call, and each time the grid is reloaded, the selection made previously is lost. If, for example, we are selecting upsell products, we could select 3 products from the grid by ticking the corresponding checkboxes. Now, we click on the ‘Next’ link to load the next page of products and perhaps, select 3 products here too. Then, we click on the ‘Previous’ link to load the previous page again. We may find that our 3 previously chosen products are no longer selected. The same would happen for sorting and filtering. So while Grid component overcomes the limitation of multi-select boxes, the features of pagination, filtering and sorting add another issue in the sense that our selected data is not preserved across these actions.

How the Serializer Block solves data preservation problem

Magento provides another block called the ‘Serializer Block’ for preserving selection data. The serializer block is normally appended to the grid block. This block adds a hidden field and creates a Serializer JavaScript object. The Serializer JS Object uses the Grid JS Object to retrieve any selected rows. Then it serializes the data i.e. key1=value1&key2=value2&… and stores this serialized string in the hidden field. As this block is always outside of the grid block, even after the grid block is reloaded via AJAX, the value of it is preserved. This serialized string is passed as part of the form submission data which can easily be decoded using the PHP function parse_str().

However, a form element doesn’t only store user input data but also displays what has been stored in it. For example, when we enter some text in a textbox, it displays whatever we have typed. When we select any option from a select box, it displays which options have been selected. Similarly, this new user input element should also display which rows a user has selected. In other words, it should mark checkboxes checked for each selected row.

For this, Serializer JS object keeps its interaction with the Grid JS object by observing events from the grid. So when a grid block is reloaded, the Serializer JS object automatically selects checkboxes of rows, which we may have already selected.

So, with this theory out of the way, we can focus on how to apply this knowledge to create our own custom admin grid interfaces.

Using the Grid Serializer Block

Magento provides a built-in library for creating custom grid and serializer blocks. Here, we can take Upsell Products grid as an example to understand how this works.

On the product edit page, there is a tab – Upsell Products. The tab content is loaded via AJAX. The action called for this AJAX tab is: Mage_Adminhtml_Catalog_ProductController::upsellAction().

So the layout update loaded for this action is defined as below in the layout XML file app/design/adminhtml/default/default/catalog.xml


    
        
        
            
                
                    catalog.product.edit.tab.upsell
                    getSelectedUpsellProducts
                    links[upsell]
                    products_upsell
                
                
                    position

Here, the root block is core/text_list type. So all children blocks are automatically rendered in sequence. See Demystifying Magento’s Layout XML Part 1.

First child block catalog.product.edit.tab.upsell is type of adminhtml/catalog_product_edit_tab_upsell, which is inherited class of Mage_Adminhtml_Block_Widget_Grid. So it creates a grid. The second child block adminhtml/widget_grid_serializer is of type adminhtml/widget_grid_serializer, which is the serializer block. So as we discussed above, the serializer block is appended to the grid block.

In layout XML, for the serializer block, an action initSerializerBlock is called with the following parameters:

grid_block_name: It defines the grid’s block name in the layout. In our case it is catalog.product.edit.tab.upsell.
data_callback: This defines a method to be called on the grid block instance to retrieve already selected values. In our case it is getSelectedUpsellProducts().
hidden_input_name: This is a hidden input element name by which the data can be retrieved from the POST object. In our case it is links[upsell].
reload_param_name: This is the parameter name by which the selection data is posted back to the AJAX call while reloading the grid for pagination, filtering, sorting etc. In our case it is products_upsell.

When the serializer block is initialized, it calls a method defined by data_callback on our grid block instance to retrieve initial data. In our case it is getSelectedUpsellProducts(). Here is the code for this method in Upsell Grid Block:

public function getSelectedUpsellProducts()
{
    $products = array();
    foreach (Mage::registry('current_product')->getUpSellProducts() as $product) {
        $products[$product->getId()] = array('position' => $product->getPosition());
    }
    return $products;
}

Here, it returns already saved upsell products. The serializer JS object stores it in the serializer field. So initially, when our grid is loaded for the first time, the serializer sets saved upsell products as selected.

In the grid block, a protected method _prepareColumns() is used to define the grid columns. The first column of upsell grid block is defined as checkbox:

$this->addColumn('in_products', array(
    'header_css_class' => 'a-center',
    'type'            => 'checkbox',
    'name'            => 'in_products',
    'values'          => $this->_getSelectedProducts(),
    'align'           => 'center',
    'index'           => 'entity_id'
 ));

Here, type is defined as 'checkbox' so the column will display checkboxes for each row. values defines the checkbox values i.e. product ids for which checkboxes should be selected. Here, $this->_getSelectedProducts() is called to get the selection data.

There is also a method getGridUrl() defined in the upsell grid block:

public function getGridUrl()
{
    return $this->_getData('grid_url') ? $this->_getData('grid_url') : $this->getUrl('*/*/upsellGrid', array('_current'=>true));
}

This method defines the URL to be called for AJAX reloading while paginating, sorting or filtering. This URL should return only a Grid block without including any other blocks such as the Serializer block.

The action method for this URL is defined as:

public function upsellGridAction()
{
    $this->_initProduct();
    $this->loadLayout();
    $this->getLayout()->getBlock('catalog.product.edit.tab.upsell')
->setProductsUpsell($this->getRequest()->getPost('products_upsell', null));
    $this->renderLayout();
}

The layout XML for it is defined as:

So the Serializer block adds a hidden field with a name as defined by hidden_input_name. When a user selects any row, the serialized data is stored in that hidden field. When a user browses next/previous pages, sorts the grid or filters by a value, the grid is reloaded via AJAX. The selection data is added as a parameter to this AJAX call with parameter name defined by reload_param_name, which is products_upsell in our case. As mentioned above in the code snippet of upsellGridAction(), the value of the POST parameter products_upsell is retrieved and passed in setProductsUpsell() on our upsell grid block instance. So calling getProductsUpsell() on upsell grid block instance will return previously selected data.

I mentioned above that the call $this->_getSelectedProducts() is used for getting selection data. Here is the code for this method:

protected function _getSelectedProducts()
{
    $products = $this->getProductsUpsell();
    if (!is_array($products)) {
        $products = array_keys($this->getSelectedUpsellProducts());
    }
    return $products;
}

As mentioned earlier, the selection data can be retrieved by calling getProductsUpsell() on the upsell block instance. Here, it is attempted to retrieve selection data by calling this method. But in case, we are editing an existing product and if, after loading the Upsell Products tab, we haven’t made any changes, the grid should display already saved upsell products as selected. However, if there is no previous selection data found, the grid attempts to retrieve any edited upsell products by calling $this->getSelectedUpsellProducts().

Finally, when the product form is submitted, the selection data can be retrieved from the POST object by the hidden_input_name key. In our case it is links[upsell]. This data is still in serialized format. Magento provides a Helper Method for decoding serialized data: Mage::helper('adminhtml/js')->decodeGridSerializedInput().

The product controller calls _initSave() before doing actual save operation. This method contains code, which is responsible for decoding serialized input:

$links = $this->getRequest()->getPost('links');
...
if (isset($links['upsell']) && !$product->getUpsellReadonly()) {
    $product->setUpSellLinkData(Mage::helper('adminhtml/js')->decodeGridSerializedInput($links['upsell']));
}

So, to summarize, the complete execution sequence of the Grid Serializer block:

Initially, Grid and Serializer blocks are loaded
When Serializer block is initialized, it loads saved selection data, stores it in a hidden field in serialized format and marks the corresponding entities as selected in the grid block.
When a user makes any changes on the grid, it is automatically reflected in the hidden serializer field.
While paging, filtering or sorting, only the grid is reloaded not the serializer. So our serializer data is not lost.
While reloading, the current selection data is passed in the POST parameters to an AJAX call.
The Grid block attempts to retrieve data from the POST. If no data is found, it retrieves the saved data.
When the form is submitted, selection data is retrieved in serialized format.
Finally the selection data is decoded and stored in the database.

Summary

Here, we have taken the example of Upsell Product management to understand how to use the Grid and Serializer blocks in combination as a replacement for multi-select boxes. In my next tutorial in this series, I will discuss more advanced usage of the Grid Serializer to create custom widgets that provide a more fully featured and user friendly interface.

When Should You Override Magento Core Files In app/code/local/Mage?

Robert Popovic — Mon, 12 Mar 2012 04:00:10 +0000

The short answer is:
NEVER!

The longer answer is somewhat less absolute and requires some explanation.

Often you will see Magento tutorials or forum posts that implement a feature or functionality change that is different from what the core does which requires changing something in the core of Magento. Someone may then advise that, instead of hacking core files it is better to copy them into the app/code/local/Mage location and perform the modifications there, so that, upon upgrade, the modifications are not lost. This process is called overriding Magento core functionality and is based on the fact that Magento sets its PHP include paths to first look in app/code/local/ then app/code/community/ and finally in app/code/core/. This has the effect that any files of the same name placed under the local or community name space will take precedence in loading, hence, we can override almost any core file in this way. To learn about how exactly Magento sets up its system, read the excellent series on the Magento Configuration by Alan Storm.

This is one, sometimes recommended, way of overriding core functionality without hacking the core but why is it, as you may conclude from my initial statement, so bad?

First, let’s see what scenarios would be compelling us to override core files:

We want to change a piece of functionality in a core method so we copy the php file containing the code and modify one or more methods.
We want to add a new method to a core block class, so it’s available for use in the phtml template so we copy the relevant core Block php file and add our method to it.
We may have several modifications consisting of core functionality changes and additions over several files.

So, what are we actually doing here, when we override core files?

For one thing, we must override the complete core file because we can’t trim out the stuff that we don’t want changed since we would lose all that functionality and most likely break Magento. Once the overridden file is in place, this will be the file and code Magento will be using from now onwards. Given that most core classes contain several and many times a large number of methods it means that we are effectively overriding all those methods in our file.

Now, remember when we were advised to use this override approach so we can make sure that our customizations are going to be preserved after a Magento version upgrade? Well that, indeed, will be true, but, what if the new Magento version has changes in the very files we have previously overridden? What if they have newly implemented methods or bug fixes in existing methods? Since our override will always take precedence, these new features and fixes will never be operational as our override runs all the old code. If you’re lucky, you may get an error report because a new change falls over due to missing methods but what if you don’t get error reports and instead, somehow, some inexplicable and erratic behavior starts expressing itself on your site?

You can see the problem now, right? After each upgrade, you will have to go and check all your app/code/local/Mage/ overrides and compare them to the new core files and port any core changes to your local override in order to maintain your site’s integrity. If you have lots of such overrides, this will be tedious. Also, if you are a developer who gets passed on an existing site with these kinds of overrides, you will not be happy. Often you won’t even know what the purpose of the override was and you’d need to laboriously go from diff to diff and decipher the meaning of any changes. It gets worse if the previous developer already performed some Magento upgrades.

By now, I guess, you have a good idea about the dangers of applying this kind of override, so, let’s get back to our original question, when, if at all, should you use this approach?

There are a few cases when this approach may be justified, in my opinion, these are the cases:

If you want to quickly try a core modification to see if it will solve your task as a proof of concept, but don’t want to create an extension just yet. When you are satisfied, you remove your override and implement a proper Magento extension with rewrites.
If you are implementing a temporary override of some core functionality that you will remove after your task is complete. A prime example of this is overriding the Magento dataflow classes for importing, when you want to change the way the importer handles certain things in an initial customer or product import.

The key concept here is: temporary changes/experiments. You are not planning on leaving your overrides.

Note: There are some other scenarios where local Mage overrides are the only way to customize parts of Magento’s core functionality. I’d like to encourage a discussion in the comments to see what others think about this.

Conclusion

The take-away from this article is, never use the app/code/local/Mage/ approach to permanently override core functionality! If you must, then only use if for temporary changes that you will remove afterwards.

As for the question that now presents itself: how, then, should you implement core overrides? The answer is by creating a custom Magento module and using the available class rewrite mechanisms, or better yet, using event observers if possible.

Quick Tip: Disable all Local Modules at Once

Robert Popovic — Mon, 13 Feb 2012 05:00:14 +0000

While going through the study guide for the Magento Developer Exam, I’ve been learning a few interesting details that are squirreled away in the Magento code, but can be useful in day to day Magento development.

For example, while going through the section on Module loading, it is revealed that we can easily disable all modules living in the local (app/code/local) namespace by editing app/etc/local.xml and setting the disable_local_modules node to true:


    
        ...
        true
        ...
    
    ...

Of course, if you have cache enable, clear or disable it to see the effects.

This will immediately prevent Magento from loading any code from your local code pool. You could use this to troubleshoot issues with custom modules or local Mage overrides. Of course, to find the exact culprit would require disabling individual modules one by one but at least you can narrow down your area of search quickly this way.

Note that this will not affect the community modules.

Quick Tip: Easy pop-ups with window.js in the front end with Magento

Robert Popovic — Sat, 05 Nov 2011 05:33:51 +0000

Magento comes with a lot of files, utilities and libraries, one of which is window.js. It is a Prototype based library implementing some classes that allow you to add windows to a web page. This can be simple pop-ups and modal windows and dialogs. They can be dragged, minimized, resized and closed. You can open multiple windows if you wanted to. You can load content from a link, an HTML container on the page or via Ajax. This library is seriously versatile. The Magento admin already uses window.js for various pop-up functions such as the widget options and image management. Well if the back end can do it, then the front end can too.

How to use it

The library is located under [magento_root]/js/prototype/window.js. Additionally, window.js comes with a bunch of CSS themes found under: /js/prototype/windows/themes/*. To load the library in the front end we will make or add an entry to our design theme’s local.xml:


    
        
        js_cssprototype/windows/themes/default.css
        js_cssprototype/windows/themes/magento.css

The bare minimum necessary to make the windows look right is default.css. In our example above, we are also loading a supplied window.js theme: magento.css. This is the theme that the back-end loads. Now, that we have the library and styles loaded, what can we do with this? For example, we can load the compare items page in a nice pop-up instead of opening a new window. Make an override of the template/catalog/product/compare/sidebar.phtml in your local design theme and add the following to the end:

Also modify the line that says:

to:

Add some items to compare and click on the “Compare” button and, voila, your compare page loads in a nice pop-up. I know, the Magento window theme isn’t exactly the most exciting one, but you can copy a theme CSS file to your local skin CSS folder and customize to your heart’s content.

Now, we might notice that the button actions in the compare window don’t quite work as expected. This is due to some of the links like Add to Cart and Add to Wishlist use setPLocation to set the target page URL. This won’t work in our pop-up since window.js loads the page into an iframe. So the remedy is to override the product/compare/list.phtml and add:

Then replace all setPLocation calls with setILocation.

The close button won’t work either but you don’t really need that as the window provides its own close button so you can simply remove it from the template.

Conclusion

In our example we saw how to open the compare page in a window.js pop-up. You can use this approach to create size guide pop-ups for example. However this is a powerful library and can do much more that just pop-up windows. To get a good overview of its power, visit the window.js homepage at prototype-window.xilinus.com. You will find the documentation and a few examples there.

Magento Sort By Newest Products Made Easy

Robert Popovic — Thu, 27 Oct 2011 09:13:04 +0000

The Situation

Recently, I had the pleasure of working on an existing site we took over. The site had been upgraded from Magento CE to EE before it came to me. One of the tasks was to make a new enterprise theme and clean up any unused or incompatible extensions.

I will just focus on one extension here that was added to allow the catalog to be sorted by newest products. This is the EWTechnologies_SortByNewest. It came to my attention after debugging the toolbar for not showing the pagination links where there is more than one page of products. So I looked into it and to my horror realized that it was overriding the default Mage_Catalog_Block_Product_List_Toolbar class but extending from Mage_Page_Block_Html_Pager. If that wasn’t bad enough, it was also completely overriding the Mage_Eav_Model_Config class. This one has been substantially changed in later versions of Magento so the override implements very old code. I decided that this is too risky to keep even though there were no other visible adverse effects. It certainly isn’t best practice to run outdated code in any case.

I thought this feature shouldn’t require overriding so much core code so I investigated and, to my delight, found that the solution was very simple and quite elegant and didn’t involve overriding any core code as such.

The Approach

The Product Entity in Magento has a ‘created_at’ attribute which stores the date when the product was created. Logically, it lends itself to be used for the sort by newest feature as it is set only once by the system, when the product was created.

A look into the eav_attribute table gives us the attribute_id and an inspection of the catalog_eav_attribute table shows that the attribute is set to not be used in the sort by drop-down. So, the obvious course of action was to set the attribute’s used_in_sort_by field to 1 and then the only other thing left was to update the frontend_label in eav_attribute to read “Newest”. That’s all, folks!

The Extension

I’ve whipped up a quick extensions that will perform those updates and makes it easy to install on any site that needs this feature.

Note: This should work fine for Magento CE 1.4.x and up, PE, EE 1.8.0 and up. There is no guarantee and I don’t provide support for this extension. Use at your own risk and ALWAYS backup your database before installing.

Download it here.

Conclusion

This approach should be upgrade safe and works on any Magento CE version over 1.4 as well as PE and EE versions. The only issue would be if a future Magento upgrade was updating the created_at attribute but then it’s easy enough to reapply the database edits to restore functionality – easier than troubleshoot issues with core code overrides.

Update: If you want to set the default sort direction for the toolbar, you can do this via the layout XML. Best in your local.xml but you can edit catalog.xml if your theme has overridden that:


    /* other layout nodes */
    
        desc
    



    /* other layout nodes */
    
        desc

Quick Tip: Using a Frontend Model For Your Module Configuration Parameter

Robert Popovic — Mon, 05 Sep 2011 05:00:44 +0000

If you’re a fastidious Magento module developer, you will try to make sure that your module options only reveal the necessary and configured fields to the admin user and hide all the irrelevant stuff. Specifically, I’m referring to Yes/No options that allow the administrator to enable and configure a subsequent option. For example, you have an extension that can show additional field content if an option is enabled and allows for entering the field contents:

Standard options configuration

As we can see, the initial state is ‘No’ for the ‘Enable Additional Text’ option, and the textarea is blank. It would be really nice if the whole ‘Additional Text’ row was hidden in this scenario.

We can achieve that by utilizing a front end model for our Yes/No option. Magento provides this mechanism for creating options that depend on certain settings or other dependencies. Some of the built in options are the ‘Allowed Countries’, Enable Flat Catalog Products/Categories and the Tax Destination Calculation ‘State’ option.

We are going to use the front end model to add some logic to enable/disable the textarea and some JavaScript to enable the toggle functionality when the Yes/No drop-down is changed. Note that this is just one, albeit trivial, example of using this technique. Especially since Magento provides the node for this specific purpose (see the update below).

Using the same tutorial module from our Explain Your Module Configuration Options With Tooltips quick tip, we’ll change our system.xml to read:



    
        
            Magebase
            700
        
    
    
        
            magebase
            Magebase Tutorial Options
            text
            10
            1
            1
            1
            
                
                    Conditional Toggle Options
                    text
                    10
                    1
                    1
                    1
                    
                        
                            Enable Additional Text
                            select
                            adminhtml/system_config_source_yesno
                            custom/adminhtml_system_config_form_field_customopt1toggle
                            10
                            1
                            1
                            0
                            Select whether to show a text block in the front end.
                        
                        
                            Additional Text
                            textarea
                            20
                            1
                            1
                            0
                            Enter the content for the text block.
                            Note that the content will be erased if you disable it above, and save.

The key to our quick tip is in:

custom/adminhtml_system_config_form_field_customopt1toggle

So, now, we’re going to create the front end model class. Incidentally, even though it’s named a ‘model’ class we are going to create it as a Magento Block class since this is how the other built in front end model classes are implemented. Effectively, it is used for template output so it’s appropriate to create a block class for this.

Create Customopt1toggle.php under .../Magebase/Custom/Block/Adminhtml/System/Config/Form/Field/ with the following content:

class Magebase_Custom_Block_Adminhtml_System_Config_Form_Field_Customopt1toggle extends Mage_Adminhtml_Block_System_Config_Form_Field
{

    /**
     * Get element ID of the dependent field to toggle
     *
     * @param object $element
     * @return String
     */
    protected function _getToggleElementId($element)
    {
        return substr($element->getId(), 0, strrpos($element->getId(), 'mbcustomopt1')) . 'mbcustomopt2';
    }
    /**
     * Get element ID of the dependent field's parent row
     *
     * @param object $element
     * @return String
     */
    protected function _getToggleRowElementId($element)
    {
        return 'row_'.$this->_getToggleElementId($element);
    }
    /**
     * Override method to output our custom HTML with JavaScript
     *
     * @param Varien_Data_Form_Element_Abstract $element
     * @return String
     */
    protected function _getElementHtml(Varien_Data_Form_Element_Abstract $element)
    {
        // If our Yes/No toggle is 'No' then disable the text value field
        if(!$element->getValue() || $element->getValue()!=1) {
            $_frm = $element->getForm();
            $_elm = $_frm->getElement($this->_getToggleElementId($element));
            $_elm->setDisabled('disabled')->setValue('');
        }
        // Get the default HTML for this option
        $html = parent::_getElementHtml($element);
        // Set up additional JavaScript for our toggle action. Note we are using the two helper methods above
        // to get the correct field ID's. They are hard-coded and depend on your option names in system.xml
        $javaScript = "
            ";

        $html .= $javaScript;
        return $html;
    }
}

The Varien_Data_Form_Element_Abstract class implements the _getElementHtml() method responsible for generating the HTML for each option element. This is where we can add out override and ‘inject’ some custom code to control the other element as well as the JavaScript to implement the dynamic toggle functionality. The code is self-explanatory.

Basically, we have a couple of helper methods to return our HTML field and row ID’s depending on the names we gave our options in config.xml. The _getElementHtml() method just sets the disabled state of the textarea field if the option is ‘No’ and adds the Prototype JavaScript to add our toggle functionality. The JavaScript code will execute on page load and completely hide the text field if the option is ‘No’ and install an event observer on the Yes/No drop-down to show/hide the text option.

The result of our work would look like this initially:

Toggle options - Text Hidden

Selecting ‘Yes’, wil toggle the next field into its visible state:

Toggle options - text revealed

Conclusion

We’ve seen how to create a nice little usability enhancement to busy module option pages. You can take this paradigm and enhance it for your own specific purposes. You can study the other examples of the built in Magento options using a frontend_model class for more inspiration and ideas.

One note, the above solution will erase a text option once its saved when the toggle is set to ‘No’. If you’d like to preserve the option, just remove the PHP code setting the text field as disabled as well as the $('{$this->_getToggleElementId($element)}').disabled = (!enabled || enabled!=1); JavaScript line. That should allow the hidden option to be saved.

Update:

The same result as above can be achieved by utilizing the node in the system.xml as Damián pointed out:


    
        Enable Additional Text
        select
        adminhtml/system_config_source_yesno
        10
        1
        1
        0
        Select whether to show a text block in the front end.
    
    
        Additional Text
        textarea
        20
        1
        1
        0
        1
        Enter the content for the text block.

However, this article illustrates how you can potentially add custom option processing using the front end model option. This is more relevant for example if you have an option where you need to check more complex conditions or link several options together.

Quick Tip: Conditional Loading of JavaScript and CSS Using Your Module Configuration

Robert Popovic — Tue, 09 Aug 2011 21:26:26 +0000

I don’t know about you but I often find myself removing third party module’s CSS and sometimes even JS files from loading in my custom themes because I need to change most of the styling and don’t want to add bulk to my site.

Traditionally, I do it using my local.xml as outlined in my previous article. However, it would be good practice if module developers would provide the option right from their module configuration.

This is actually really easy to do, watch this:

1. Create configuration options in system.xml:

...

    
    Advanced
    text
    100
    1
    1
    0
    
        
            Load Stylesheet
            select
            adminhtml/system_config_source_yesno
            10
            1
            1
            0
            Set to No if the supplied stylesheet file should not be loaded.
        
        
            Load Javascript
            select
            adminhtml/system_config_source_yesno
            20
            1
            1
            0
            Set to No if the supplied javascript file should not be loaded.
        
    
    

...

Here, I’m only showing the relevant part for brevity. Essentially, I have an Advanced section with two Yes/No drop downs which allow the admin to configure the desired options.

2. Add your conditional includes to your module’s layout XML:

        
            css/my_module/my_module.css
            skin_jsjs/my_module/my_module.js

And there you have it. Your css and js will only be included if the configuration allows it.

Improving the File Cache Backend

Vinai Kopp — Mon, 01 Aug 2011 21:00:44 +0000

The purpose of this post is to spread knowledge about the bolts and gears of the Magento cache system among developers, and to share one method of overcoming some limitations of the file cache storage class.

This article basically started with the site of a client who was having performance issues: when the cache reached about 38000 records, he was actually forced to clear the cache to keep the site responsive enough.

How strange is that? Shouldn’t a full cache give a better performance then an empty cache?

The number of records stored in the Magento cache depends on many factors, amongst others the number of store views, if a full page cache is used or not, and the size of the catalog. Many Sites don’t go above 1000 or 2000 cache records, but for large instances much higher values are common.

The performance issues of my client only occurred on specific pages, one of the most noticeable being the add-to cart action. Dropping a product to the cart took up to 4 seconds!
Using some profiling, we could pin the issue down on part of the (full page) cache being cleared, more precisely, when the mini-cart in the page header had to be rebuilt.

Lets have a look at how the Magento caching (which uses the Zend_Cache library) works.
Each cache entry consists of the following information

the cached data
a cache key (or ID), that uniquely identifies this entry and is used to retrieve the data from the cache
a cache lifetime, after which the cache entry expires
zero or more cache tags

Most of these are obvious, but what is the purpose of cache tags? Cache tags are used to segment the cache for partial deletion. For example, you could clear all entries of the configuration cache, without touching the any entries of the HTML block cache.

On the cache management page most of the cache tags used by Magento are listed. Depending on the modules and extensions installed there could be more or less tags, e.g. CONFIG, LAYOUT_GENERAL_CACHE_TAG, BLOCK_HTML, TRANSLATIONS, FULL_PAGE_CACHE, …

Listed Cache Tags

All the information associated with the cache record is saved in the cache storage every time the method Mage::app()->saveCache() is called. And to read cache records the Method Mage::app()->loadCache() is used.
Magento offers several different options what to use as a cache storage, and each of these storage systems is used by means of a PHP class called “cache backend”.

By default, cache data is stored in files (located in the directory var/cache/). Another option is the database, which is slower then the files option mostly because the database is a heavily used resource for Magento instances anyway. Then there are storage schemes that use RAM (which are much faster), e.g. APC (shared memory) or memcached (a networked caching server).

So, lets use RAM as our cache storage, right?

Sounds good, except for one problem: APC and memcached only support storing simple key-value pairs, so the cache tags are lost! This renders the whole caching rather useless, because every time we need to clear only one part of the cache, EVERY cache entry is cleared.

But, do not despair! The Zend Framework contains a solution to this problem. There is a special cache backend called Twolevels. The Twolevels backend uses a fast cache backend (i.e. APC or memcached) for the cache data, and a slow backend (i.e. files or database) for the lifetime and the cache tag information. This way we can have the best of both worlds!

According to the excellent performance whitepaper (login required) Magento has released, it is recommended to use APC for the fast backend and files for the slow backend for single webserver setups. If you use a cluster of webservers, you should use memcached for the fast backend and the database for the slow backend. The latter has more overhead then the APC/files combo, but having all servers share one cache pool makes up for that.

Magento makes it easy to configure all this by the way – have a look at the file app/etc/local.xml.additional for further information. I will not go deeper into the setup here because we have already written about this.

Now, back to the problem at hand – why is a nice, full, cache slower then an empty one?
The site in question was using APC as the fast backend and files as the slow backend.

So, lets have a look at what is happening.
Every time a product is added to the cart, the following method is called:

Mage::app()->getCache()->clean(
    Zend_Cache::CLEANING_MODE_MATCHING_ANY_TAG,
    array($cacheTag)
);

$cacheTag is an MD5 hash built from several parameters identifying the active customers mini-cart block cache.
So, how do the twolevel and the file cache backends handle this?

In the method Zend_Cache_Backend_TwoLevels::clean() each request with the cleaning mode CLEANING_MODE_MATCHING_ANY_TAG fetches all cache IDs matching the given tags from the slow backend using the method getIdsMatchingAnyTags(), and then removes one cache entry after the other in a loop.

case Zend_Cache::CLEANING_MODE_MATCHING_ANY_TAG:
    $ids = $this->_slowBackend->getIdsMatchingAnyTags($tags);
    $res = true;
    foreach ($ids as $id) {
        $bool = $this->remove($id);
        $res = $res && $bool;
    }
    return $res;
    break;

Lets dive in a little bit deeper and find out how the Zend_Cache_Backend_File backend finds those cache IDs.
The interesting part happens in the method _get(). Here we can see a list of all cache entries is built, and then it loops over each one, reads in the corresponding meta data file (using file_get_contents() and unserialize()) and then checks if the cache entry is associated with a matching cache tag.

This following code is slightly simplified for educational purposes:

$result = array();
$glob = @glob($cacheDir . $prefix . '--*');
if ($glob !== false) {
    foreach ($glob as $file) {
        $fileName = basename($file);
        $id = $this->_fileNameToId($fileName);
        $metadatas = $this->_getMetadatas($id);
        …
       switch ($mode) {
           …
           case 'matchingAny':
               $matching = false;
               foreach ($tags as $tag) {
                   if (in_array($tag, $metadatas['tags'])) {
                       $result[] = $id;
                       break;
                   }
               }
               break;
           …
       }
    }
}

This code is probably fine for a couple of hundred cache entries, but after not even a day my clients Magento instance reached 40k records. Opening and unserializing thousands of files takes some time, even on a strong machine.

It turns out the file backend doesn’t scale well.

The remedy to this is obviously some kind of index, mapping the tags to the matching cache IDs.
Instead of writing that information into a file that would have to be parsed and updated, I decided to use the filesystem itself as the index utilizing directories and symlinks.

In the modified file cache backend developed for the client, every time a cache entry is written, a directory with the name of each tag is created and a symlink to the metadata file is placed into it. This gives a little more overhead during the write operation. There is no difference reading cache entries. But every operation matching some tags is a lot faster. And since Magento makes heavy use of cache tags, the effect is quite noticeable, depending on the number of records in the cache. For example, adding a product to the cart now takes less then a second on the original instance. And I’m happy to say the reports of the friends who have been nice enough to test the extension have been positive for smaller sites also.

Benchmarks!

Inspired by Marc Jakubowski’s comment below, I added a little benchmark script. Here are some results to give a more detailed idea.
UPDATE: Thanks to Collin Mollenhour’s comment below, I also added benchmarks for Redis using his Zend Cache Backend Class – it’s incredibly fast, so if you have the possibility to use Redis 2.4 or newer, go for it! For this test I used the default Redis configuration, with the server running on the same machine I was running the tests. Before using it in production I would like to write some unit tests for this new backend, but it seems to work okay.

Records	Tags	Avg. Records / Tag	Cache Backend	Avg. Time for getIdsMatchingTags()
50000	250	~700	File	20.71s
50000	250	~700	Symlink	2.28s
50000	250	~1500	Redis	0.01s
6000	120	~175	File	2.41s
6000	120	~175	Symlink	0.23s
6000	120	~370	Redis	0.002s
2000	80	~88	File	0.78s
2000	80	~88	Symlink	0.08s
2000	80	~180	Redis	0.001s

The numbers for the records and tags where chosen because they roughly correspond the average cache pool size and tags on several live Magento instances of different sizes. These benchmarks where run on my laptop, and obviously the results may be different depending on the drive, file system, system memory etc. Please go ahead and test on your system.
One interesting thing about the Redis backend is that the average number of ID’s per tag is much higher then with the other backends. I’m not sure why, my guess is that it has something to do with the way Redis manages the tag hash tables. I don’t have time to check it out at the moment, but maybe Collin already knows why.

By the way, according to the PHP reference, the symlink function is no longer limited to Unix systems, but since PHP 5.3 it is also available under Windows (Vista, Server 2008 or greater). I haven’t tested this, though.

This module is provided with no warranty, you use it at your own risk. I know it is being used successfully on several sites, both as a primary cache backend and as a slow backend in combination with APC or memcached.
I invite you to have a look at it and try it out, but please start with a test instance and not your live store. You can download the module from Magento Connect or from github.
If you find bugs or have improvements, please send them in.

After installing the extension clear the config and block_html cache and visit System > Tools > Symlink Cache.

There you can see sample XML that you will have to add to your app/code/local.xml file for both variants, to use it on its own or to use it as a slow backend.
Then, after you have updated the local.xml file, clear the config cache, go to the Symlink Cache page and hit the “Initialize Tag Symlinks” button.
And that is all, your system is set up and uses the tag symlinks. Enjoy!
I would be happy to hear about other methods, ideas and experiences about improving cache performance.

Downloads

Magento Connect

Github

Google Optimizer on Magento: Where to start for maximum gain

Mark Hammersley — Mon, 01 Aug 2011 02:23:05 +0000

Magento has Google Optimizer tools built in and so it makes sense to use them. Google Optimizer allows you to present different content options to your prospective customers and then tracks which version converts more. On top of this Google Optimizer has built in probability formulae that understand when a result is a ‘confident result’ rather than a random occurrence.

To get a confident result you need to either have a test that is really obviously better or a decent number of transactions. But as a rule of thumb if you are getting more than 10 transactions a day then you can benefit from using Google Optimizer.

Where to start?

If you have a lower number of transactions it makes sense to start optimizing on the pages that most people see. This tends to be the home and checkout pages of the site. However we need not guess the best place to start when we have good data available to tell us exactly.

We can find the best place to start by looking at Google Analytics. Google Analytics should be setup using Ecommerce mode and to be tracking conversions, which is a standard part of Magento as long as you have entered the Google ID in the Google API area of Magento’s configuration settings.

In Google Analytics go to ‘content’

Then top content

Then set the time line for Google Analytics to show a decent length of time. I am using 6 months in my example – just use as much data as you have since the site has been stable and not changed.

On the top content screen find the right most column called $ index – the $ index is the average value for a page that a user visited before completing an Ecommerce transaction. Ordering by the $ index will give you the most valuable pages on your site. However you also need to click on ‘weighted sort’ to give you not only the most valuable pages but the most relevant valuable pages. Without weighted sort you are only going to see the pages that randomly got high values because an odd buyer used them to make a big purchase. By using weighted sort we can filter out these anomalies.

On the site I am looking at for this example this gives me the following pages

As you can see these are all checkout pages which are NOT part of the inbuilt Google Optimizer set of tools that Magento supplies. Whilst I have personally run split tests here, its tricky and not something I can easily show you. However you can optimise these pages by creating a ‘goal’ funnel and looking at the profile, but I want to cover that in another blog post.

So what I am going to do is to scroll through the list until I find one of the following pages that is built into Magento to Google Optimizer test

Product page
CMS page – (this could be a Google Adwords landing page and in many of my sites these are the best pages to test as they drive a significant proportion of my revenue)
Category page
Home page

In my example site the first page I find is not the home page as I would have expected but instead a product page. It just goes to show how important it is to look at the stats before deciding where to test.

The product I can use to test is worth $102 and thus a decent sale for the site and must have a fair number of transactions to rank this highly. So this is where I would start.

I am not going to go through how to actually setup the experiment as there are plenty of good blog tutorials available – just Google “Google Optimizer for Magento” – but I wanted to show you where to test first as this will actually allow you to make the most important changes to your site first.