Monday, August 15, 2011

Use Migrate Module to Format Incoming Data

This week's post will be brief and focus on a simple example of formatting some data on it's way through the Migrate module.

In my situation, I will be using Migrate for sync'ing some data from a MS SQL server db into a couple of Vocabularies in my Drupal environment. Unfortunately, my limitations on this db are such that I can't write a view to do this work for me, and I won't dare "clean" up some data for my needs.

So, I am sitting on a few rows of data that do not come across due to the use of a n-dash character which causes an error when attempting to save into the MySql db. My solution is to address this data issue in one of my Migration classes to clean the data on it's way in. I will be inserting the prepareRow function into my class as noted in the drupal docs:


I will be sure to highlight my big mistake because I overlooked a key detail in the article:

"The argument $row is a stdClass object containing the data as provided by the source"

When I first wrote my code, I was referencing the destination field name which never yielded any errors. I just didn't do anything. I didn't realize my folly until I was stepping through the code and I inspected the values directly. So remember, use the source column names! Last but not least, I expanded beyond just the n-dash character and replaced for a larger set of unfriendly characters:

 function prepareRow($row) {

$row->Category = strtr(
$row->Category,
array(
"\x80" => "e", "\x81" => " ", "\x82" => "'", "\x83" => 'f',
"\x84" => '"', "\x85" => "...", "\x86" => "+", "\x87" => "#",
"\x88" => "^", "\x89" => "0/00", "\x8A" => "S", "\x8B" => "<", "\x8C" => "OE", "\x8D" => " ", "\x8E" => "Z", "\x8F" => " ",
"\x90" => " ", "\x91" => "`", "\x92" => "'", "\x93" => '"',
"\x94" => '"', "\x95" => "*", "\x96" => "-", "\x97" => "--",
"\x98" => "~", "\x99" => "(TM)", "\x9A" => "s", "\x9B" => ">",
"\x9C" => "oe", "\x9D" => " ", "\x9E" => "z", "\x9F" => "Y"));
}

Hopefully the simplicity of this post doesn't disappoint, but this solution took me longer than it ever should have. I hope this post helps someone else from making the same mistake!

No comments:

Post a Comment