Switch from AWStats to Piwik on a Webfaction host

Posted on Thu 17 January 2013 in misc

Webfaction provides AWStats as a default, "one-click" webapplication. The ease of installation (it uses only the Apache logs) and use is however a trade off to more powerfull analytic systems such as Google Analytics, or Piwik.

This last package is an free/open source equivalent of Google Analytics, and allows you to keep your own data on your own server. If you like to have full control of your website(s) and are not comfortable with e.g. the idea that you are sending the IP-addresses and surfing behaviour of your visitors to some US multinational, it is worth a look.

If you would like to switch from AWStats to Piwik (or run them simultaneous), the following steps should get you started.

Installing Piwik

First create a "Static"-application in your Wefaction panel and a website so that this application is accessible. I will assume you call the application piwik, and that you configure a website at myname.webfactional.com/piwik/. The website location will be the base URL for Piwik, which is also contained in the tracking-snippet.

Navigate to the newly created directory of the application, download Piwik and extract the files.

cd ~/webapps/piwik/
curl -O http://builds.piwik.org/latest.zip
unzip latest.zip
mv piwik/* .
rm -r piwik

Point your browser to the previously defined Piwik base url, e.g. http://myname.webfactional.com/piwik/index.php, and follow the straightforward installation steps. You will need to create a MySQL-database (and user) using the Webfaction panel. The entire installation should take about two minutes, and you should have an empty, running Piwik-installation after that. You can select the generated tracking-snippet during the installation (or from the "Websites"-section afterwards) and insert it unmodified in your webpages/templates.

Importing older logs

We would like to import the older visitors logs into our new Piwik-installation. Webfaction unfortunatly only keeps logs for 7 days, but it is better then nothing, and you can use the same process for longer logs that you perhaps have kept yourselve. Installing Piwik will not conflict with your AWStats application, so you can use them side by side, and view historical data in AWStats. And no, it is not possible to import the older AWStats-data into Piwik, it is summarized in an AWStats-specific format.

First step is combining the different and rotated logs into one file. You can use a standard AWStats-tool that is present in you AWStats application folder, which I assume is called awstats. Wefaction stores the logs in one directory, we only want the logs for one website, called mysite in this example.

cd ~
# change the name, but do keep the stars
./webapps/awstats/tools/logresolvemerge.pl logs/frontend/*mysite* > import.log

Optional: you can activate the GeoIP functionality before you import the logs, it does not seem possible to reprocess the imported data if you activate the plugin afterwards.

Piwik provides a Python script to import older logs. You will need to provide the correct base URL for the Piwik installation, and possibly a different site id, if you are not configuring your first/only site. The other options are optional, e.g. wether you import the logs for bots or not.

# single line/command
python webapps/piwik/misc/log-analytics/import_logs.py 
    --url=http://myname.webfactional.com/piwik/ dump.log --idsite=1 
    --enable-reverse-dns --enable-http-errors --enable-http-redirects 
    --enable-static --enable-bots

If you screw up something on import, and wish to clear the imported data for the site without reinstalling you fresh Piwik-setup, you can accomplish this with a few SQL-statements.

Navigate to the database you have set up for Piwik in your Webfaction panel, select phpMyAdmin and log in with your Piwik database username and password. Select the "SQL"-tab and paste the following SQL-statements in the text field (the date-part of the table will be different).

DROP TABLE piwik_archive_numeric_2013_01, piwik_archive_blob_2013_01;
DELETE FROM piwik_log_visit WHERE idsite = 1

The site id would be 1 if it is a fresh install, and the dropped tables will be recreated on your first visit.

Your Piwik installation is now populated with the older visitor logs, and will be collecting data as soon as you embed the Javascript-snippet.