Schneimi’s Dev Weblog


Batch generated spritemap and css with ImageMagick on Windows

Posted in CSS,Eclipse by schneimi on February 3, 2013
Tags: , , , ,

I had the idea of one click generated spritemaps for quiet a while, but never had time to look into it and kept going on manually creating them by online service for my projects. When I read an article about command line css spriting, I was amazed, how easy the map can be built using ImageMagick and that it’s even possible to read width and height from the images. Nevertheless, the automated generation of the css code was still missing. Especially when the icon order changes, this is still annoying work.

This made me think…we know how to create the spritemap and can also read the width and height of each sprite with ImageMagick, so the only thing that is left, is to cycle through the sprites, read width or height from them and write the css code to a file. Then I dig out some of my Windows batch skills and came up with the following little batch script. It’s not that easy to understand, so I made some comments on the relevant lines.


spritemap.bat

@echo off
rem /* prepare environment */
setlocal EnableDelayedExpansion
del spritemap.png
del spritemap.css
set position=0
set sign=
rem /* execute imagick convert and create a horizontal (use -append for vertical) sprite map, using black as transparency color */
convert.exe *.png -transparent black +append spritemap.png
rem /* loop through all png files */
for /f %%f in ('dir /B *.png') do (
  rem /* for each file execute imagick identify */
  for /f "tokens=3 delims= " %%i in ('identify.exe %%f') do (
    rem /* parse the image width from the result (use tokens=2 for height in a vertical sprite map) */
    for /f "tokens=1 delims=x" %%j in ('echo %%i') DO (
      rem /* for each image append this css code to the spritemap.css file */
      echo .ui-%%~nf { background-image: url^("../img/spritemap.png"^)^; background-position: !sign!!position!px 0^; }>>spritemap.css
      rem /* calculate the next background position */
      set sign=-
      set /a position=%%j+position
    )
  )
)

In my case the css is for custom icons in jQuery Mobile. This example only runs with all files in one folder, but paths can be adapted easily, as well as the css output. Also further image optimization with tools like optipng, as described in the mentioned article, can be easily added.

I Hope this makes someone as much happy as I are, finally having found a fully automated solution. For me it’s a perfect solution, because I already use batch-scripts to merge and pack all css and js files with one click in eclipse and I just had to add this script to my existing batch file.

Advertisements

User specific caching in CakePHP

Posted in CakePHP by schneimi on February 2, 2013
Tags: , ,

CakePHP caching depends all on controllers and given arguments. But, what if you want different cache files for conditions that don’t come along with the arguments, like session variables?

In my case, I simply wanted separate cache files for each user of my app, because some actions deliver different views for different users, and I didn’t want to pass the User.id to each action in every controller that was concerned.

So I had a look into the CacheHelper and noticed that the file name for cache files is just build from the controller name and given arguments. First I tried to make a modified version of CacheHelper for my app. I don’t remember what problems I ran into, but it just didn’t work out very well.

Then I came up with the idea to internally manipulate the URL, just before the cache is read and written. Somehow this lead me to the lib/Cake/Routing/Filter/CacheDispatcher.php, where I needed just a few lines of additional code to put the User.id into place, which worked like a charm and had no side effects.

You have to copy the original lib/Cake/Routing/Filter/CacheDispatcher.php into your app app/Routing/Filter/UserCacheDispatcher.php and add the following lines of code. To put the dispatcher into action you finally have to register it in the bootstrap.php and replace the core CacheDispatcher.


app/Routing/Filter/UserCacheDispatcher.php

(...)
public function beforeDispatch($event) {
    // load SessionComponent
    App::uses('SessionComponent', 'Controller/Component');
    $session = new SessionComponent(new ComponentCollection());
    $userID = $session->read('User.userID');

    if (!empty($userID)) {
      // add user id to request url with 'u' prefix
      $event->data['request']->here = $event->data['request']->here.'/u'.$userID;

      // remove request parameters to solve issue with ajax request random number parameter
      $event->data['request']->query = '';
    } else {
      return;
    }
(...)


app/Config/bootstrap.php

Configure::write('Dispatcher.filters', array(
    'AssetDispatcher',
    'UserCacheDispatcher'
));

I wonder why this use case is not supported by CakePHP out of the box, or didn’t I just see it? Well, I am aware that this might not be the best solution, but it is at least a pretty simple and effective workaround. Any ideas on how to solve the problem in better ways are welcome.

CakePHP Base64 encoded caching file size

Posted in CakePHP by schneimi on February 2, 2013
Tags: , , ,

I stumbled upon the problem that some cached files in CakePHP took alot more space than expected, and I read about a similar problem on StackOverflow

Well, I took a deeper look into this and it looks like the serialized data in viewVars is used for the nocache parts of the view. So the solution should be to place the part where the base64 data is output between nocache tags.


<!--nocache--><?php echo $base64Data; ?><!--/nocache-->

But doing this, I had the effect, that on some views it worked, on others it didn’t. A closer look into CakePHP revealed, that preg_match_all() is used to look for the nocache parts, which cannot handle a very large amount of data (may depend on server settings) and only finds nocache parts up to a certain amount of data. It doesn’t even throw an error, wich I checked with preg_last_error(). I read about rising pcre memory limits, but it didn’t work for me, so I had to find another solution for that problem.

In my case image data was retrieved from database and placed base64 encoded within the view in an image src tag (src=”data:image/png;base64,…”). My solution was to replace the data in the image src tag with a link to an extra action that delivers just the image data and caches it seperately. This also has the advantage, that the image can now be cached by browsers apart from the view.

The first thing I did, was to care about the image data not coming with the find, in order not to get serialized in the viewVars. But I still needed the information about if there actually was an image in the data, which brought me to setup a virtual field in my model. The virtual field does the following, if there is image data present in the field it holds 1 (TRUE), otherwise 0 (FALSE).


app/Model/YourModel.php

public $virtualFields = array(
    'image' => 'IF(image IS NOT NULL, TRUE, FALSE)'
);

http://book.cakephp.org/2.0/en/models/virtual-fields.html

For the image to retrieve and use, I setup an extra action without view in my controller and cached the data manually. Unfortunately you also have to care about deleting it when your model gets deleted or updated, but you can easily do that in the model save and delete callbacks.

http://book.cakephp.org/2.0/en/models/callback-methods.html


app/Controller/YourController.php

public function image($id) {
  $this->autoRender = false;
  $this->cacheAction = false;

  // read data from cache
  $data = Cache::read('image_'.$id);

  if (empty(data)) {
    $data = $this->YourModel->find('first', array(
        'conditions' => array(
            'YourModel.id' => $id
        ),
        'fields' => array(
            'image_type'
            'image_data'
        )
    ));

    // write data into cache
    Cache::write('image_'.$id, $data);
  }

  // send cache header for browser caching
  header('Content-Type: '.$data['YourModel']['image_type']);
  header('Cache-Control: public, max-age=28800');

  // output the raw image data
  echo $data['YourModel']['image_data];
}

In the view the image src looks like this:

<img src="/your_controller/image/<? echo $id; ?>" />

Another idea would be to generally unset the viewVars if nocache parts aren’t used in a view. But I don’t know where to hook in for that.

Trac SubticketsPlugin with progress bar and report

Posted in Trac by schneimi on February 2, 2013
Tags: , , ,

I came across the problem, that I needed a sub-ticket plugin for Trac v1.0. I found the SubTicketPlugin for TRAC on trac-hacks.org but also the ChildTicketsPlugin, that comes along with a very neat progress meter for parent tickets, but unfortunately is not yet available for Trac v1.0.

Because I wanted it so much, I searched for a way of having such a progress meter with the SubTicketPlugin. Then finally I found a way of building a site template for Trac, where the progress meter is displayed conditionally. It’s not that easy to understand and also uses some inline python code to cover all conditions, but it should work without further explanation. Just place the site.html in the /templates folder of your trac project.

ticket_view


/templates/site.html

<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:xi="http://www.w3.org/2001/XInclude"
      xmlns:py="http://genshi.edgewall.org/"
      py:strip="">
<div py:match="div[@class='description']" py:attrs="select('@*')">
  ${select('*')}
  <?python
    subticketCount = 0
    if 'ticket' in vars() and ticket.id != None:
      from trac.env import open_environment
      env = open_environment('D:\Development\TRAC\myapp')
      with env.db_query as db:
          cursor = db.cursor()
          cursor.execute(''.join(["SELECT count(*) FROM subtickets WHERE parent=", str(ticket.id)]))
          for row in cursor:
            subticketCount=row[0]
  ?>
  <py:if test="subticketCount > 0">
    <h3>Progress </h3>
    ${wiki_to_html(context, ''.join(["[[TicketQuery(format=progress,parents=", str(ticket.id), ')]]']))}
  </py:if>
</div>
</html>

I also developed a custom report, showing the tickets and their sub-tickets.

ticket_query

SELECT p.value AS __color__,
       id AS ticket,
       (SELECT GROUP_CONCAT(summary) FROM ticket WHERE id = parent) AS __group__,
       summary,
       component, version, milestone,
       t.type AS type,
       owner, status,
       CONCAT(ROUND(100 / (SELECT COUNT(*) FROM subtickets WHERE parent=t.id) * (SELECT COUNT(*) FROM subtickets INNER JOIN ticket ON (child=ticket.id ) WHERE parent=t.id AND status='closed')), "%") AS progress,
       CASE WHEN (SELECT COUNT(*) FROM subtickets WHERE parent=t.id) > 0 THEN 'border-bottom:solid 3px #DDD;border-top: solid 3px #DDD;background-color: #DDD' ELSE 'text-indent: 10px' END AS __style__,
       time AS created,
       changetime AS _changetime,
       description AS _description,
       reporter AS _reporter
  FROM subtickets s
  LEFT JOIN ticket AS t ON (parent =  t.id)
  LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority'
  WHERE status <> 'closed'
  GROUP BY ticket
UNION
  SELECT  p.value AS __color__,
       id AS ticket,
       (SELECT GROUP_CONCAT(summary) FROM ticket WHERE id = parent) AS __group__,
       summary,
       component, version, milestone,
       t.type AS type,
       owner, status,
       CONCAT(ROUND(100 / (SELECT COUNT(*) FROM subtickets WHERE parent=t.id) * (SELECT COUNT(*) FROM subtickets INNER JOIN ticket ON (child=ticket.id ) WHERE parent=t.id AND status='closed')), "%") AS progress,
       CASE WHEN (SELECT COUNT(*) FROM subtickets WHERE parent=t.id) > 0 THEN 'border-bottom:solid 3px #DDD;border-top:solid 3px #DDD;background-color: #DDD' ELSE 'text-indent: 10px' END AS __style__,
       time AS created,
       changetime AS _changetime,
       description AS _description,
       reporter AS _reporter
  FROM subtickets s
  LEFT JOIN ticket AS t ON (child = id)
  LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority'
  WHERE status <> 'closed'
  GROUP BY ticket
ORDER BY __group__ DESC