Utah Open Source Planet

Selenium, Nagios and Magento– together at last!

2010-07-29T16:51:26+00:00

Now that I have a server running selenium tests against a magento-powered ecommerce site and logging those tests to a file, we need to integrate with nagios. First, I installed openSSH on that windows server (via copssh), and copied over the nagios user public key to authorized_keys. Then I wrote this nagios script:

#! /usr/bin/php -q
<?php
ini_set( 'max_execution_time', '55' );
ini_set( 'display_errors', false );
 
define( "STATUS_OK",       0 );
define( "STATUS_WARNING",  1 );
define( "STATUS_CRITICAL", 2 );
define( "STATUS_UNKNOWN",  3 );
 
$execCommand='ssh WINDOWS_SERVER_IP_GOES_HERE -l nagiosremote "tail -n 1 selenium.txt"';
exec($execCommand,$output);
list($id,$date,$time,$status,$avg,$std,$FLAG,$flagCnt,$statCnt)=split(',',$output[0],9);
if ((strpos($status,'OK')===false)||(strpos($FLAG,'ERROR')!==false))
{
        echo("ERROR: $status $FLAG\n");
        exit( STATUS_CRITICAL );
}
else
{
        echo("OK: time $time, status $status, avg $avg, std $std, FLAG $FLAG, flagCnt $flagCnt, statCnt $statCnt\n");
        exit( STATUS_OK );
}

Then I defined a check_command, a host, a host_group and a service in nagios and we’re in business!

Final week to submit papers for UTOSC 2010!

2010-07-29T03:18:56+00:00

This Monday marked the beginning of the final week to submit papers for the 2010 Utah Open Source Conference.

If you miss this final chance to submit a paper you may well miss out on your opportunity to present!

In addition to being a presenter there are several other opportunities to contribute to the Utah Open Source Conference, such as volunteering your time or conducting a BoF get together.

Check out http://blog.utos.org and http://2010.utosc.com for all the latest opportunities to help out this year!

Lest we forget

2010-07-28T22:48:50+00:00

I have not worked with a Windows Vista computer in some time. I forgot how bad it sucked. Now, let us qualify sucking. I have never used a computer that has a respectable hardware stat-sheet, and been so dog slow in all of my life.

Case and point: Today I am using a Toshiba Laptop that has a Intel Pentium Dual-Core 1.6 GHz processor and 2 GB of ram. Now while these two figures alone do not give us a perfect picture of expected performance any machine with these components should be just fine.

I clicked the start orb, then on control panel. Ten minutes later the control panel came up. Ten minutes. I sat there waiting for just the control panel. I then selected ‘Network and sharing’, another 3 minutes. After it finally opens all looks fine there, I now need to add a printer. Go back to Control Panel. After 7 minutes of just sitting there with that blue donut whirling it finally did something. Yes folks, a flash of a blue screen and the computer restarted.

I wish that I could take this computer and do one of two things, install something else, like my favorite free operating system, or downgrade to Windows XP or upgrade to Windows 7. But alas, it is not my computer. It is the personal computer of a co-worker who needs access to certain things on our network. I won’t go into the pain of this running windows Vista Home Premium and that connecting even to drives in our active directory is a hack at best. That will be a post for another day.

Remembering in KRL: Using Entity Variables with Forms

2010-07-28T21:33:53+00:00

Dan asked a question in the Kynetx Developer Exchange about remembering user entered data in KRL. I gave him a brief outline of the solution but thought an example would be nice. This blog post is the detailed answer to how to gather, remember, and use user-supplied data.

The basic idea is to store the data in an entity variable. Trails (a type of persistent variable) are the most appropriate type of entity variable to use. The ruleset pattern has four rules: initialize, send the form, process the form, use the data. The actual ruleset has five because I added one to delete the user data since it makes testing much more convenient. I'll go over each rule in order.

Forget: The first rule just clears the entity variable ent:name when you visit a particular web page. You could, of course, do this under user control with a form submission or something, but this is easy enough and suits my purpose for testing.

rule clear_name is active {
   select when web pageview "www.foobar.com" 
   noop();
   always {
      clear ent:name;
      last
   }
 }

Note that if the rule is selected (the page URL matches) then the entity variable is cleared and this is the last rule executed in this ruleset.

Initialize: I like initializing the area of the page I'm going to write or modify and then do the modification in later rules since I can have different rules do different things to the area as needed. So the initialization rule just puts up an empty notification box with a div named #my_div that we'll use in late rules.

rule initialize is active {
  select when pageview ".*"
   pre {   
    blank_div = <<
<div id="my_div">
</div>
    >>;
  }
  notify("Hello Example", blank_div)
      with sticky=true;
}

Note: for all the rules in this ruleset that are selected by web pageviews, I've made the URL pattern as general as possible (.*). In a real ruleset, these would likely be much more restricted.

Send the Form: This rule puts the form into the div we initialized in the last rule if the entity variable ent:name is empty (we could probably add a primitive test that makes this easier). The rule also sets a watcher on that form. If this rule fires (i.e. the rule is selected and the action premise is true) then this will be the last rule executed in this ruleset.

rule set_form is active {
  select when pageview ".*"
  pre {   
    a_form = <<
<form id="my_form" onsubmit="return false">
<input type="text" name="first"/>
<input type="text" name="last"/>
<input type="submit" value="Submit" />
</form>
    >>;
  }
  if(not seen ".*" in ent:name)  then {
    append("#my_div", a_form);
    watch("#my_form", "submit");
  }
  fired {
     last;
  }
}

When this rule fires, you get a notification box that looks like this:

Process the form: We need a rule to process the form. This rule is selected when the form is submitted. The rule doesn't have an action (i.e. the action is noop();). All the real work is done in the postlude where we store the name in the entity variable ent:name and then raise an explicit event that will cause another rule to be selected.

rule respond_submit is active {
  select when web submit "#my_form"
  pre {
     first = page:param("first");
     last = page:param("last");
  }
  noop();
  fired {
     mark ent:name with first + " " + last;
     raise explicit event got_name
  }
}

Use the data: The final rule uses the data in the entity variable to put the user's name in the div we placed in the initialization rule. This rule has two selection conditions. It can be selected on a pageview like the other rules or when an explicit event named got_name is raised. Remember that the previous rule raises that event when it's done. The action replaces the contents of the div having the ID my_div with a hello message that includes the name.

rule replace_with_name is active {
   select when explicit got_name 
            or web pageview ".*" 
   pre {
     name = current ent:name;
   }
   replace_inner("#my_div", "Hello #{name}");
}

When this rule fires, the notification box looks like this:

Whenever this ruleset is selected in the future the user will not see the form but simply see this box because the system remembers the name in the entity variable and has no need to ask for it again. If the data gets cleared, then the user is prompted to enter it again.

This ruleset also shows the use of explicit events. If we don't raise the explicit event got_name in the rule that stores the name, nothing will replace the contents of the notification box and show the user that the form submission is successful. We could have done it in that rule, but then we'd have two rules replacing the contents with a message and if the message changed we'd have to make sure we changed it in two places. This technique allows us to have one rule responsible for putting the hello message in the div. We just fire it under two different circumstances.

Update: Here's a video that shows this ruleset in action:

Tags: pds kynetx krl krl+examples

Quick Selenium RC Runner Script (updated)

2010-07-27T22:31:10+00:00

define('BADWEIGHT',10);
define('GOODWEIGHT',1);
define('BADMULTIPLIER',150);
define('BADTIMEOUT',10);
define('SLOWDEVIATIONS',2);
$filename = 'C:\Program Files\ICW\home\nagiosremote\selenium.txt';
 
$execCommand="phpunit.bat selenium.php";
$trial=0;
function logToFile($somecontent)
{
	global $filename;
	if (is_writable($filename))
	{
		if (!$handle = fopen($filename, 'a'))
		{
			 echo "Cannot open file ($filename)";
			 exit;
		}
		if (fwrite($handle, $somecontent) === FALSE)
		{
			echo "Cannot write to file ($filename)";
			exit;
		}
		fclose($handle);
	}
	else
	{
		echo "The file $filename is not writable";
	}
}
 
function sd_square($x, $mean) { return pow($x - $mean,2); }
 
function sd($array)
{
   return sqrt(array_sum(array_map("sd_square", $array, array_fill(0,count($array), (array_sum($array) / count($array)) ) ) ) / (count($array)-1) );
}
 
function getStats($trials)
{
	foreach($trials as $trial)
	{
		$sum+=$trial['time'];
		$timeArray[]=$trial['time'];
	}
	$counter=(count($trials)>0)?count($trials):1;
	$average=sprintf("%.2f",$sum/count($trials));
	if (count($trials)>1)
	{
		$stdDev=sprintf("%.2f",sd($timeArray));
	}
	else
	{
		$stdDev=0.00;
	}
	return array($average,$stdDev);
}
 
function stripOutliers($times)
{
	$noOutlierArray=array();
	list($avg,$stDev)=getStats($times);
	foreach($times as $lineItem)
	{
		if ($lineItem['time']<=($avg+$stDev))
		{
			$noOutlierArray[]['time']=$lineItem['time'];
		}
	}
	return $noOutlierArray;
}
$flagCnt=0;
$statCnt=0;
while(1)
{
	$output=array();
	exec($execCommand,$output);
	$time=intval(trim(substr($output[4],6,3))); //parse seconds
	list($status,$hud)=split(' ',$output[6],2); //parse status
	$status=trim($status);
	if ($status!='OK')
	{
		$status=implode("-",array($output[6],$output[8],$output[9],$output[10]));
		$statCnt+=BADWEIGHT;
	}
	else
	{
		$statCnt-=GOODWEIGHT;
	}
	$resultArray[$trial]['time']=$time;
	$resultArray[$trial]['status']=$status;
 
	list($avg,$stDev)=getStats(stripOutliers($resultArray));
	$FLAG=($time>($avg+SLOWDEVIATIONS*$stDev))?"SLOW":'OK';
	if ($status!='OK') $FLAG='WARNING! Bad Status ';
	if ($FLAG=='SLOW')
	{
		$flagCnt+=BADWEIGHT;
	}
	else
	{
		$flagCnt-=GOODWEIGHT;
	}
	if ($flagCnt<=0) $flagCnt=0;
	if ($statCnt<=0) $statCnt=0;
	if ($flagCnt>(BADWEIGHT*BADMULTIPLIER)) $FLAG='ERROR! TOO MANY SLOW TRANSACTIONS';
	if ($statCnt>(BADWEIGHT*BADMULTIPLIER)) $FLAG='ERROR! TOO MANY TRANSACTION FAILURES';
	if ($FLAG=='ERROR! TOO MANY TRANSACTION FAILURES'&&$status=='OK') {$FLAG='OK';$flagCnt=0;}
	if ($FLAG=='ERROR! TOO MANY SLOW TRANSACTIONS'&&$status=='OK') {$FLAG='OK';$flagCnt=0;}
	$lineItem="${trial},".date('d/m/Y H:i:s').",$time sec,$status,Avg:$avg,StdDev:$stDev,$FLAG,$flagCnt,$statCnt\n";
	echo($lineItem);
	logToFile($lineItem);
	if (($FLAG!='OK')||($status!='OK')) sleep(BADTIMEOUT);
	$trial++;
}

which will produce a text file like this:

C:\xampp\php>php runTests.php
0,29/07/2010 12:06:01,34 sec,OK,Avg:34.00,StdDev:0,OK,0,0
1,29/07/2010 12:06:20,19 sec,OK,Avg:26.50,StdDev:10.61,OK,0,0
2,29/07/2010 12:06:39,19 sec,OK,Avg:19.00,StdDev:0.00,OK,0,0
3,29/07/2010 12:06:58,18 sec,OK,Avg:18.67,StdDev:0.58,OK,0,0
4,29/07/2010 12:07:17,18 sec,OK,Avg:18.50,StdDev:0.58,OK,0,0
5,29/07/2010 12:07:36,19 sec,OK,Avg:18.60,StdDev:0.55,OK,0,0
6,29/07/2010 12:07:55,18 sec,OK,Avg:18.50,StdDev:0.55,OK,0,0
7,29/07/2010 12:08:14,18 sec,OK,Avg:18.43,StdDev:0.53,OK,0,0
8,29/07/2010 12:08:38,24 sec,OK,Avg:19.12,StdDev:2.03,SLOW,10,0
9,29/07/2010 12:09:12,24 sec,OK,Avg:19.67,StdDev:2.50,OK,9,0

UTOSC 2010: Call for Papers ends Sunday, August 1

2010-07-27T04:42:37+00:00

Hi all,

Just a quick reminder to all of you out there who might be considering submitting a paper for the Utah Open Source Conference coming up this October 7-9. The call for papers is coming to a close on August 1 so hurry up and get it submitted :) I’m planning on submitting a paper for FreeSeer and will likely also present on the State of UTOS at the conference.

Cheers,

Herlo

Summer Of Pie: Sour Cream Lemon

2010-07-27T04:00:03+00:00

You might have been wondering where all the pies went. Don't fear as they are not gone for good. The memory card with all the pictures on it went on vacation to the coast (it took my wife and kids too), and with everybody gone I didn't feel up to eating an entire pie by myself. But now that they've returned, I'm back in full swing.

Just before they left I made this delicious pie. It was one of the pies listed on Shari's menu and it sounded intriguing, since sour cream isn't typically a dessert food. The recipe is fuzzy since it's been a while. Essentially what I made was a lemon curd with the addition of sour cream. I took the curd off the heat too soon and it didn't set up enough. Gun shy about curdling it I suppose.

The flavor was over-the-top lemony. Using fresh lemons helped there, I'm sure. It could have used a good bit more sugar though, as it was quite sour as well. The kids were not quite as thrilled by this one; the oldest even turned down a second piece. Just not in line with their sweet teeth, but I did enjoy it and my wife loved it. To each his own.

Changing the News with Kynetx: Utah Company WILL Revolutionize Internet Use

2010-07-26T19:08:25+00:00

Yesterday Tom Harvey of the Salt Lake Tribune had a great article on Kynetx in the Money section (see Utah company looks to revolutionize Internet use). I loved the article, but what I loved even more was what Casey Holgado (@caseyholgado), one of our summer interns, did with it. Here's the headline for the online article as it appeared:

Casey wrote a quick Kynetx app that changed the headline to read like this:

This to me embodies the spirit of Kynetx: "the Web, your way." We didn't change the text on the server, of course, only the headline for people who have the app installed in their browser. But that's the point: if you want this headline to read our way, then you install the app. Everyone else gets the default view. Welcome to Kynetx, where the user is in control.

Tags: kynetx press

Photos from Bountiful Fireworks

2010-07-24T17:19:27+00:00

Below are some photos that I took with my Canon Digital Rebel XTi prior to and during the fireworks on the 23rd.

The other photos all ended up blurry because it was dark. But these were pretty good.

FISL, Day 2

2010-07-24T15:41:31+00:00

Day two of my trip to FISL (Friday) started out with me waking up a bit late and scrambling to get ready on time. Rafael kindly picked several of us up at the hotel and drove us to the conference. I spent the morning talking with people both in the Fedora booth and in several of the other booths around the show. I also spent some time in a press interview with Linux Magazine. For lunch, we ate at the buffet and I got to experience the famous banana sushi first hand.
In the afternoon, I cheered Dennis Gilmore on as he gave a presentation on RPM packaging in Portuguese. He doesn’t speak much Portuguese (yet!), but he was brave enough to translate his talk into Portuguese and read it as best he could. After Dennis did his talk, I ran over to the large presentation room for my presentation. I talked about how Red Hat and Fedora work together, how Fedora makes a good upstream for RHEL, and how Fedora works with upstream communities. Several hundred people attended the presentation, and I thought it was well received.

After the presentation, I ran into a Brazilian friend of mine doing VoIP work here in Brazil. We chatted for an hour or so about various topics including Fedora, VoIP, and cultural differences between Portuguese-speaking and Spanish-speaking countries in Latin American. After another stint at helping out in the booth, we all ran back to the conference hall for a great presentation on SELinux from Douglas and Marcelo from Red Hat. By the time their presentation was done, it was 9:00pm and we were all hungry. Luckily, some of the Fedora folks had organized a pizza party, so we weren’t hungry for long!

FISL, Day 1

2010-07-24T00:55:55+00:00

Day one of the FISL conference (at least for me, as I arrived on Thursday) was mostly spent in airplanes. My flights took me from Dulles to Orlando to São Paulo to Porto Alegre. (Nothing is better than having a flight attendant wake you up at 3:00am to feed you and airline dinner, right?) When I landed in Porto Alegre, Leo and Rafael were kind enough to pick me up and drive me to the FISL conference. I was impressed with the size of the conference and the variety of the booths.

The Fedora booth at FISL

I have to admit — I was a bit stunned when I saw the Fedora booth. It looked absolutely gorgeous! I had no idea when I walked in whether we were sharing Red Hat’s booth or would have our own; little did I know that we’d have the best looking booth at the show. The location of the booth was perfect — we were right on the corner where people would pass by on their way to the big lecture hall.

After shaking hands with the Fedora ambassadors and volunteers and saying hello to the folks in the Red Hat booth, I gave a quick impromptu speech (in Spanish, by request) to the crowd that had gathered around our booth, and then sat back and watched as Toshio gave a great talk on packaging RPMs for Fedora, with a lot of additional explanation in Portuguese from Rafael and Leo. (We had several other presentations right there in the booth as well over the course of the day.) At that point, we had about fifty people gathered around the booth, so we handed out media and stickers and helped answer questions.

Crowd around the Fedora booth

After the show wound down for the day, we went to dinner at a traditional Brazilian churrascaria. The food was delicious — I ate too much food, and forgot how much I like the Brazilian soft drink called Guaraná. It was the perfect way to end the day. On the way back to the hotel, I got to see a bit of the city of Porto Alegre.

2010-07-23T16:36:56+00:00

The energy harvesting bandwagon continues. It's good to see so many new applications for harvesting the energy in the world around us and saving just a few more batteries from the landfill. These replacements for AA and AAA batteries from Epson caught my attention. Mechanical energy harvesting at it's simplest. Just give your remote control a good shake before you press the buttons...

Alfresco: Max Version Policy

2010-07-23T06:56:13+00:00

As part of a POC for a customer I was asked to write an extension that allowed them to control the total number of versions allowed per versioned content. (Download links at bottom of page)

Alfresco has a strong versioning story, that gives you the ability to version any content stored in the repository, no matter what the file type. Versions are full files and not diffs of the files. Alfresco gives you the ability to have both major and minor versions of content. Versions can be created/updated by checkout/checkin, by rule, through any interface or through script/APIs.

Alfresco also provides the ability to apply behaviors / policies to content/metadata within the repository. You can think of these as event listeners, that allow you to take custom actions based on what is happening within the repository. Jeff Potts has written an excellent tutorial on creating behaviors, with examples for both Java and javascript. (Other resources include: )

There are 4 versioning policies that we can work with:

beforeCreateVersion
afterCreateVersion
onCreateVersion
calculateVersionLabel

(For an example of a calculateVersionLabel policy look at this post and the accompanying code by Peter Monks.)

For this policy we are going to use onCreateVersion: when a version is created we want to remove any version that puts us past the max version value.

You start by implementing the policy interface for the policy you want to apply.

public class MaxVersionPolicy implements OnCreateVersionPolicy

Adding the method implemented by the interface

public void onCreateVersion(
				QName classRef,
				NodeRef versionableNode,
				Map versionProperties,
				PolicyScope nodeDetails)

We also need to bind the behavior to the policy and we need to do this when the class is loaded by the springframework. We wrap this registration in an init() method

	public void init() {
		this.onCreateVersion = new JavaBehaviour(this, "onCreateVersion",
				NotificationFrequency.TRANSACTION_COMMIT);

		this.policyComponent.bindClassBehaviour(QName.createQName(
				NamespaceService.ALFRESCO_URI, "onCreateVersion"),
				MaxVersionPolicy.class, this.onCreateVersion);
	}

The init method is then called when spring loads the bean

		<bean id="maxVersion"
                      class="org.alfresco.extension.versioning.MaxVersionPolicy"
                      init-method="init">
			<property name="policyComponent">
				<ref bean="policyComponent" />
			</property>
			<property name="versionService">
				<ref bean="versionService" />
			</property>
			<!-- The max number of versions per versioned node -->
			<property name="maxVersions">
				<value>10</value>
			</property>
		</bean>

Now to the meat of the policy, enforcing the removal of versions.

First we we have our maxVersion property. This is set in the springbean (see above) and read when the bean is loaded. (You can overwrite the default value by copying the maxversionpolicy-context.xml file into the extensions directory and changing the value of the maxVersion property)

	public void setMaxVersions(int maxVersions) {
		this.maxVersions = maxVersions;
	}

Next we want to remove any version that puts us over the max

 	@Override
	public void onCreateVersion(QName classRef, NodeRef nodeRef,
			Map versionProperties, PolicyScope nodeDetails) {

		VersionHistory versionHistory = versionService
				.getVersionHistory(nodeRef);

		// If the current number of versions in the VersionHistory is greater
		// than the maxVersions limit, remove the root/least recent version
		if (versionHistory.getAllVersions().size() > maxVersions) {
			logger.debug("Removing Version: "
					+ versionHistory.getRootVersion().getVersionLabel());
			versionService.deleteVersion(nodeRef, versionHistory
					.getRootVersion());
		}
	}

We first get the versionHistory for the node and check it against the maxVersion property. If we have more versions than the limit we delete the least recent version from the version history.

Notice: 10 total versions; least recent version is 1.1; no pagination!

Q: Why are you using an if statement, instead of looping through all of the versions. I have a few (many) more versions over the limit I want to impose?

A: A while statement would be more efficient in removing everything above the limit, but the versionHistory object isn’t being updated with each delete in a loop (I tried) until the policy has completely run. Luckily, as you will find as you implement custom behaviors, they get called a lot! So while a single update will result in a single version added, you will actually see that the behavior is called multiple times for that one action (In testing I saw the behavior being called a minimum of 7 times). While you might not clear out every version over the limit with a single update, you will see a good many of them removed. If you want to bring every down to the limit you could create a “cleaner” extension that could go in and remove all of the versions over the limit (this would be a very intensive/costly operation depending on the amount of content in your repository and the size of your version history – There several different strategies for this). You could then use this extension to enforce that limit.

Q: Does this affect all versioned content in the repository? What if I only want it to work on some content?

A: Yes, by default this policy is being applied to all versioned content, but you could add in checks to look for a specific aspect, parent space name, property, etc. before passing through the delete code.

Q: Compliance regulations/laws don’t allow me to delete the versions, but require me keep them for X years. How can I archive versions?

A: There are probably several different ways to handle this. One way could be to use Alfresco’s content store selector. The content store selector allows you to configure multiple underlying filesystem locations for your content. To the end user all of the content appears to come from the same location, while the content itself lives in different disc systems. The idea would be to set up a secondary store that would act as the archive. When your trigger is reached, be it a property (status [property or aspect], date, etc.), a version number/count, etc. have the policy move the content to the archive space structure (The archive will need to have the cm:storeSelector aspect set to mark it to use the secondary store). Because we are working from the least recent version, the 1.0 version of the content, it becomes the first version in the archive, each subsequent version, is added as a version for the content, version numbers are then maintained in the archive. I would also add an aspect to the original node that contains a pointer/aspect to the archived version, for reference. Then delete the least recent version of the original versioned node.

Google Code Project: http://code.google.com/p/alfresco-maxversion-policy/

Download Alfresco Max Version Policy AMP

Tested on Alfresco (Enterprise) 3.2 – 3.3.1

jQuery: Novice to Ninja

2010-07-22T23:55:25+00:00

I was recently given the opportunity to fly to Florida and hire people to clean up oil from the BP spill. Given that I had some long flights (I live in Salt Lake City, UT) I picked up a Nook from Barnes and Noble and had on my computer a copy of jQuery: Novice to Ninja. What better way to spend my flight time than reading a jQuery book! So I dug in.

I have been using jQuery both professionally and personally for about a year now but this book still brought a lot to the table.

Earle Castedine and Craig Sharkie have a great sense of humor, and obviously know their way around jQuery, CSS, HTML and the likes. The book uses some real world examples of creating image galleries and user interfaces. They break down the jQuery code to it's basic forms, and then explain why things work the way they do. They don't get too deep into Javascript except to note that jQuery is nothing more than a fancy framework on top of Javascript and uses Javascript for everything! I thought it particularly neat when they would show the "hard" way to do things, then the better way to do them, and then finally a plugin that made all the work you had just done seem useless, but it isn't useless because you now understand how the plugin works better than if you had just slapped it in place in the beginning.

The book is quite deep about animations, AJAX actions, DOM manipulation and the sort. Even if you know Javascript and you have been using jQuery there is something in here that will make you go "ah ha!".

To the folks over at SitePoint (an O'reilly company) good work!

How Different Groups Spend Their Day - Interactive Graphic - NYTimes.com

2010-07-22T22:23:00+00:00

This graph is neat-o

How Different Groups Spend Their Day - Interactive Graphic - NYTimes.com

How to get a fast response from me

2010-07-22T17:58:59+00:00

I’m swamped with work. I can’t yet talk much about the work I’ve been doing until we go public, but it’s a cool project, although its coolness may not be immediately obvious at first.

Meanwhile, I have been rather slow to answer some emails. Here is what you need to do to get a fast email response from me: present me a viable option that allows someone other than me to do the work. For example, if you want one of my open source projects changed, the best thing to suggest is that you will do the work and all I have to do is review a patch. (Ideally, include the unified diff in the email so I can review instantly.)

AV Security Suite Malware Removal: Panda said I was clean, ComboFix disagreed

2010-07-21T23:44:52+00:00

"There's this security software that keeps popping up and I can't use my computer."

Say goodbye to several hours of your life should you decide to help.

But I've got this great malware removal guide from Maximum PC, and I've been doing tech support since the mid-1990s - no problem right?

Oh, the pain. I'm going to try mitigate others' potential suffering by sharing what helped me.

The Maximum PC Guide for removing malware contains some great comprehensive steps. Here are a few things that can be done to supplement the guide.

Booting the operating system that has the problem can slow you down at first.

1. Erase a spare USB Key and use UNetbootin to create a bootable Kaspersky Rescue Disk. Use this tool to remove some of the less resilient threats.

2. Boot in safe mode and run the following from a CMD prompt to delete Temp & "Temporary Internet Files" directories for all user Profiles(assumes XP install w/profiles on C: drive ):

C:\> for /f "usebackq" %a in (`dir /b /a "c:\documents and settings"`) do rd /s /q "c:\documents and settings\%a\Local Settings\Temp"

C:\> for /f "usebackq" %a in (`dir /b /a "c:\documents and settings"`) do rd /s /q "c:\documents and settings\%a\Local Settings\Temporary Internet Files"

3. Defrag. Auslogics has a good free defragmenter, or you can use the built-in defragmenter.

4. Autoruns

from Mark Russinovich and Bryce Cogswell(Don't use this unless you feel very familiar with windows services) - Click the Options menu, Check the Boxes for "Hide Microsoft and Windows Entries" and "Verify Code Signatures" as shown in this screenshot:

Push the Escape key, then hit F5 to refresh in Autoruns. Now inspect the list and right-click the odd looking entries. You can choose to "Delete" the offender or "Jump To..." to open up a Window with the startup location referencing the item - usually in the registry.

5. Run CCleaner. Cut Deep - but remember to be nice - some users like their Internet History kept intact, and I like to keep my Start >> Run history while troubleshooting.

6. At this point you can go ahead and step through the Maximum PC guide. What follows is the snag I ran into in removing a piece of malware called AV Security Suite and how to get around that snag thanks to some specific posts from chaslang at MajorGeeks and other posts from BleepingComputer.

After running SuperAntiSpyware and Malwarebytes Anti-Malware, and subsequently running Panda's Activescan 2.0, it looked like the machine was clean. I then started ComboFix and received the following error, but ran ComboFix anyway: Panda FAIL. AV Security Suite was supposed to be gone. Panda said the box was clean! Why, Panda, why? The ComboFix log displayed the following AV Security Suite GUID:

AE716D16-40FE-4cb9-8FD2-2975088F55B2

Googling that GUID resulted in a lifesaver post by chaslang, which stated there was a Browser Helper Object(BHO) that sill needed to be deleted. I skipped the first suggestion in chaslang's post to run MGTOOLS\analyse.exe, but followed his other instructions.

HijackThis found the offender.

I was able to click Fix and afterward run ComboFix according to chaslang’s instructions without ComboFix detecting the AV Security Suite. Chaslang, I owe you... something. Thank you!

When rebuilding or updating a home machine connected to the internet, I strongly suggest using the automated, toolbar-free installers created by http://ninite.com/ Makes it easy to install Microsoft Security Essentials, Flash, FireFox, Foxit Reader, Auslogics Disk Defrag, and several other apps. Ninite will save you a lot of time.

Another troubleshooting suggestion is to boot Windows off a CD or USB key(WinBuilder and Boot Land can help you create a bootable version of windows, but these are deep waters that will take some time to navigate for first-timers) and thereby work on the infected system in a safer environment.

Testing Magento Sites with Selenium

2010-07-21T19:08:29+00:00

So you installed the Selenium IDE firefox plugin and you recorded a check-out transaction of your Magento-powered ecommerce site. Then you save as a php file and fire up your Selenium RC server and run the script. It gets to the checkout page and … Nothing. What’s wrong?

What’s wrong is that you’ve submitted too quickly. You need to do some checking with xpath ids for the “updating totals” div before you can click submit. Here’s a working Selenium Magento script for your reference:

require_once 'PHPUnit/Extensions/SeleniumTestCase.php';
 
class Example extends PHPUnit_Extensions_SeleniumTestCase
{
 function setUp()
 {
   $this->setBrowser("*chrome");
   $this->setBrowserUrl("http://www.example.com");
 }
 
 public function testMyTestCase()
 {
   //$this->maxRunningTime= 30; //(this is a special function I implemented)
   $this->open("/");
   $this->click("//ul[@id='nav']/li[1]/ul/li[1]/a/span");
   $this->waitForPageToLoad("30000");
   $this->click("//img[@alt='ADD TO CART']");
   $this->waitForPageToLoad("30000");
   $this->click("//div[2]/div[1]/ul/li/a/img");
   $this->waitForPageToLoad("30000");
   $this->type("billing:email", "test@example.com");
   $this->type("billing:firstname", "test");
   $this->type("billing:lastname", "test");
   $this->type("billing:telephone", "8015551212");
   $this->type("billing:street1", "100 Main Street");
   $this->type("billing:city", "Salt Lake City");
   $this->select("billing:region_id", "label=Utah");
   $this->type("billing:postcode", "84116");
   $this->click("//*[@id=\"billing:street1\"]");
   for ($second = 0; ; $second++) {
       if ($second >= 7) break;
       try {
           if (!$this->isElementPresent("//span[@id='billing-please-wait' and contains(@style,'display: none')]")) break;
       } catch (Exception $e) {}
       sleep(1);
   }
 
   for ($second = 0; ; $second++) {
       if ($second >= 7) break;
       try {
           if ($this->isElementPresent("//span[@id='billing-please-wait' and contains(@style,'display: none')]")) break;
       } catch (Exception $e) {}
       sleep(1);
   }
 
   $this->type("merchant_cc_number", "511111111111111");
   $this->click("advice-validate-cc-number-merchant_cc_number");
   $this->type("merchant_cc_number", "4111111111111111");
   $this->select("merchant_expiration", "label=01 - January");
   $this->select("merchant_expiration_yr", "label=2016");
   $this->type("merchant_cc_cid", "432");
   $this->click("//*[@id=\"submit-order-button\"]");
   sleep(1);
   $this->click("//*[@id=\"submit-order-button\"]");
   $this->waitForPageToLoad("30000");
   $this->waitForPageToLoad("");
   try {
       $this->assertTrue($this->isTextPresent("Thank you for your purchase!"));
   } catch (PHPUnit_Framework_AssertionFailedError $e) {
       array_push($this->verificationErrors, $e->toString());
   }
 }
}

Here is the Selenium timeout/maxRunningTime modification code explained.

Utah Sunsets

2010-07-21T15:47:46+00:00

I made two photographs on my way back from Salt Lake City last night. I was leaving a friend’s house after dropping off some items for the SLLUG meeting tomorrow night. While descending the avenues, I passed by Memory Grove near 9th Avenue where I stopped for an amazing sunset.

Here are several. The first two are the pictures I made last night and were made on my Nikon D300 camera. The first is looking out at Antelope Island to the West with my Nikon AF-S 24-70mm f/2.8 G ED lens and the second is a view of the Utah state capital and the sunset taken with my Nikon 10.5mm f/2.8 DX Fisheye. (All images Copyright © Marc Christensen, All rights reserved.)

I’ve taken several over the years, here are some more of my favorite Utah sunset photos I’ve made. Click on the images to see larger versions:

I love Utah.

Nikon D70, Canon 20D or wait for the Nikon D200?>>> UPDATE Oct. 28, 2005: Check out the new leaked...
Walking tour of downtown Salt Lake CityWent with some friends: Melody, Jason and Natalie on a...
Modeling work with MichelleMichelle 2, originally uploaded by Mecworks. Spent an evening in...

Theory: In each chat there are two conversations

2010-07-21T14:30:14+00:00

So I have this theory that in each chat session that we have with someone there are always two conversations going. Maybe always is a bit broad. Lets say, all of mine, and my guess is most of yours. I find that it is because of this phenomena that most effective communication can break down in a chat. Largely due to confusion on what the context is of a specific piece of the communication.

Lets look at a completely hypothetical example:

A: Hey, hows it going? Haven’t chatted in a while.
A: Wanted to ask you if you still had that car.
B: Hey, hows it going? Yeah, it has been a while.
B: What are you up to?
B: Yeah I have the car, Why?
A: I am fine, just up to the usual, Work.
A: boss is really on my nerves.
A: Was wanting to know if you wanted to sell the car.
B: Oh man that sucks! I hate my boss. Work is the pits.
B: I wish I could quit my job.
A: Oh man I know. Work is awful.
B: So you like the car? I need the money, I could sell it.
A: I hate it, it is always on my nerves.

It is at this point that confusion comes in. By looking right at the chat it looks like person A hates the car and it is always on his nerves. We can look at the two conversations seperatly and see that each is a logical, straightforward conversation.

A: Hey, hows it going? Haven’t chatted in a while.
B: Hey, hows it going? Yeah, it has been a while.
B: What are you up to?
A: I am fine, just up to the usual, Work.
A: boss is really on my nerves.
B: Oh man that sucks! I hate my boss. Work is the pits.
B: I wish I could quit my job.
A: Oh man I know. Work is awful.
A: I hate it, it is always on my nerves.

A: Wanted to ask you if you still had that car.
B: Yeah I have the car, Why?
A: Was wanting to know if you wanted to sell the car.
B: So you like the car? I need the money, I could sell it.

Each is a self contained conversation.

Has anyone else noticed this in their own chats? It seems that I am not totally alone in my findings, there is a facebook group about it. It seems too the only way to not have it happen is to wait what seems like an obscenely long time for each question to be asked and answered back.

JOYBOND | Other "Tama-chan whole."

2010-07-20T21:24:00+00:00

Be sure and check out the translated version of this page.

JOYBOND | Other "Tama-chan whole."

UTOSC 2010 Call for papers extended through August 1st!

2010-07-20T17:09:59+00:00

Time goes by quickly, so quickly in fact that top priorities get set to the side and suddenly become missed opportunities. Fortunately submitting a proposal for a presentation at the 2010 Utah Open Source Conference doesn’t have to be one of them!

The deadline for paper submission has been extended until August 1st, 2010, just in time to sit down inside a nice cool coffee shop, sip some of the great local roast and think about what to share with the community.

Really it only takes a minute or two to place a submission and you might find it so easy and fun that you submit more then one.

Don’t worry about not having a full presentation prepared, all that needs to be submitted by the deadline is the proposal, a basic outline of what would be presented to the community at the conference. The next step is easy: sit back, relax and wait to hear if your proposal was selected by vote for the conference. Once selected the entire presentation(s) should be laid out and created.

Don’t have anything to talk about? That’s not what I’ve heard!

Topics can be as wide ranging as are the members of the Utah Open Source community at large (and that is pretty gosh darn diverse!) and should include a variety of experience levels. While one presenter may submit a presentation on Building a Better Community Through Statistics another (and an excellent presentation I attended in 2009,) might be Advanced GIMP: The Digital Diet.

So don’t be afraid to submit a proposal to talk about your favorite open source program(s) that just does silly things or maybe does nothing at all!

Has anyone ever submitted a presentation on an open source pet rock? No? Hhmm…

All ideas are welcome, but please remember to keep with the 2010 theme: its better when its free*.

Being free is so easy that my open source pet rock was free!

Interested? Want more details? Visit the Utah Open Source Conference website at http://2010.utosc.com and when your ready to submit your proposal simply create an account and visit http://2010.utosc.com/speaker/papers/ to submit your paper.

You could put it off again, but then you will have to pay to attend and where’s the fun in that?

Android App Review: Setting Profiles

2010-07-20T14:30:17+00:00

Introducing a ‘new’ blog segment where I put my thoughts on some of the best, or worst, of the android apps that I have tried. Today on the docket, Setting Profiles.

We all are on the go, we move everywhere. That is why we have a mobile phone. But I find that I have a handful of places that I am at more often than others. At these various spots I have different services or settings that I use on my phone. At home and work I want to be connected to the WIFI. At low battery state on the phone I want to shut everything off to conserve battery. When at a funeral service, or Sunday while at church I want the phone completely silent. Conversely, when not at home, I don’t want WIFI on depleting my battery needlessly. Or when not at church I want to hear my phone. What is a real pain in the butt is to set these things at each new location every time.

In steps Setting Profiles. There are other apps that also change phone settings based upon certain phone criteria, but I found them cost prohibitive. Settings Profiles only cost $3.95 and is so similar in feature set for my needs it is a no-brainer to save the money.

With Setting Profiles there are some things that very first must be done after installing the app. First, set up some Locations.When you go to add a new location it gives you a couple options, I always chose Lat/Lon which then gives you a map. You simply select where, the distance in radius that you must be from this point, and then a name. Viola, you have set a location. The really cool thing, you don’t have to be at the location, you just have to select ANY point on the map.

Next is to set profiles. When setting a profile you will then select what things should be on/off. Just go down the list and turn things on and off as needed.

Last step, is to set up a rule saying that if you are in location A turn on setting B. Once your location criteria is met Setting Profiles will change your phone settings. Once you no longer meet the location criteria, Setting Profiles will change your settings back to how they were before.

But what if you want something to happen and it is not dependent upon location. Never fear! There are many criteria choicesthat you have. Battery State, Calendar appointment title, Missed call, Blue Tooth device connected, on even just a specific scheduled time to name a few. Do you want your phone to automatically silent from 8:00 PM to 8:00 AM so you don’t get disturbing calls in the night? No problem, this can do that.

What if you don’t want it to change phone settings? You do have the option of Running a program or Notifying you also.

Last points to highlight. You also can select multiple profiles at a time. It simply applies them in order. If your first rule turns on WIFI and the second one disables it, WIFI will be off. Another neat feature is the widget which allows you to quickly use aprofile, disable a profile, or use a profile on a timer. Very convenient for that meeting to be an hour. Time the profile setting with the widget for one hour and automagically your phone settings change when you are done. The widget also shows you the order that the profiles are applied in so that you can more easily refine your rule-set.

What I like: I like the convenience of just knowing that my phone settings will change when they are supposed to without me thinking about it. Very convenient to know that stuff is just magically working for me. Also, the UI is actually quite simple to use and very intuitive.

My wish list: It seems that the GPS location rules are stuck at 1/2 mile. I can’t get a 1/4 mile to work at all. I wish that all the granular options worked for this. Also, on the Calendar Items condition my only option is that for 12 hours prior to the calender item I can have some action happen. Really? 12 hours? With all our that this app can do we only get the ‘choice’ of 12 hours? I want to see options for 12 hours, 1 hour, 30 minutes, and 5 minutes at LEAST. With a more finite time interval I could set it so that when I had a funeral, or had a family arrangement or meeting of some kind it could auto silence my phone. 12 hours is just rediculous. I would also like to see a cell phone tower location option for CDMA phones. Maybe there is a technology limitation there, not sure. But it would still be nice.

All in all: I would still recommend this app to people. Not just as a cheaper alternative than Locale, but as a great option for criteria based phone settings changes. The price point is such that it really is a good value, and really, my gripes are minor. Also, the app is still under active development, which means that improvements should continue to happen (should being the operative word).

Laptop to an AP

2010-07-19T14:30:20+00:00

So, the other day I was at a location that has a wired internet connection but no wireless and no switch. I didn’t have switch and wireless with me and so, I decided to take on the task of making my laptop a wireless access point with WPA2 security. The guide that I followed the most is here, and much thanks to the author, although I did find that I had to do more to get it to work than the guide provided.

Very first you will need to have installed the following packages:

wireless-tools
iw
linux kernel 2.6.30 or later (I am using 2.6.31-21 ubuntu generic)
hostapd
dhcp-server

My hardware: Macbook Pro 3,1 running Ubuntu Linux (9.10 Karmic). This laptop has a wireless card that uses the atheros ath9k driver.

The rest of this article assumes that you already have a good working WIFI card, my point is not to troubleshoot why your wifi card doesn’t work. Also I assume that you know how to set up a DHCP server, know how to edit files, etc.

First, after making sure that my wireless card would even do it I set the static IP on my wirless card.

ifconfig wlan0 192.168.10.1 netmask 255.255.255.0

Next, I set up DHCP on the wireless interface only with my dhcpd.conf looking like this.

subnet 192.168.10.0 netmask 255.255.255.0 {

range 192.168.10.10 192.168.10.49;

option routers 192.168.10.1;

option ip-forwarding off;

option subnet-mask 255.255.255.0;

option broadcast-address 192.168.10.255;

option domain-name-servers 4.2.2.2, 4.2.2.1;

}

Then I set up my hastapd.conf file thusly.

interface=wlan0

driver=nl80211

ssid=YourSSID

channel=6

hw_mode=g

auth_algs=1

wpa=3

wpa_passphrase=12345678 (Change this, obviously)

wpa_key_mgmt=WPA-PSK

wpa_pairwise=TKIP CCMP

rsn_pairwise=CCMP

Lastly, run the thing.

#hostapd -d hostapd.conf

Some other caveats. I did have to change some iptables stuff to allow routing through the eth0 interface and allow access to the DHCP server through port 67. My rule set looked like this:

-A POSTROUTING -o eth0 -j MASQUERADE

-A INPUT -p tcp -m tcp –dport 67 -j ACCEPT

-A INPUT -p udp -m udp –dport 67 -j ACCEPT

-A FORWARD -i wlan0 -j ACCEPT

Otherwise, that was all there really was to it. My brother-in-law could connect to the web through my laptop and I could use it with the wired. The world was right as rain.

FUDCon Santiago, Day 2

2010-07-17T17:27:08+00:00

Day number two of FUDCon Santiago started with a quick breakfast at the hotel, and then we walked over to Universidad Central. Along the way, several folks stopped to take pictures — I think we even lost a couple of the Brazilians along the way. Some of the pictures have been posted to Flickr. Along the way, we ran into several phone booths that were advertising (the ever-nebulous concept of) cloud computing. In this case, they were simply providing some disk space if you signed up for their internet access, and we joked that maybe they should advertise it as “fog computing” instead. It was a bit of a wake up call, however, for just how quickly things are advancing in the area.

The first talk I attended was a talk by Guillermo Gómez S. (gomix) on RPM packaging. He did a great job of explaining the basics, and then getting into a few of the trickier subjects. After the talk, we talked to a couple of developers that were interested in having their software packaged for Fedora, and Guillermo invited them to attend his afternoon session where they actually packaged up some software.

I then jumped on IRC to participate in the Fedora Board Meeting. Máirín Duffy has written a very good writeup of the meeting on her blog. We experimented a bit with this meeting by making it totally open, and it went fairly well, but was a bit confusing at time when there were three or four conversations happening at once. We’ll keep exploring the best way to provide openness and transparency, while maintaining order as well.

After the board meeting, I bounced around from talk to talk and spent a lot of time listening to people here. There were several good talks on virtualization and high availability. I also spent more time conversing with the attendees and answering questions.

After the show wrapped up for the day, we held an impromptu round table discussion with the FUDCon organizers and the Fedora Ambassadors that were here. It was a good chance to discuss the things that went well, that didn’t go so well, and many other topics. The discussion was very animated and there were many different topics discussed, and I’m not sure I can do it all justice in this blog post, but I’m going to try to enumerate some of the topics discussed.

The first topic we discussed was communication among the ambassadors, especially between countries. It was generally agreed that the ambassadors aren’t communicating well with ambassadors from other LATAM countries, and we talked about ways we can improve the discussion between the different countries. We also talked about better ways of announcing events such as FUDCon, so that we get the word out to all the different Linux-based communities in the region. The general consensus was that wikis are great for documentation, but they don’t do a good job of letting people know of upcoming events, and that there needs to be more work done to reach out to users and developers in other ways.

The next topic was one of lack of organization within the LATAM open source community. We talked about the fact that this is a collaborative community, and that there needs to be more self-organization and collaboration between the disparate communities. At this point, it was suggested that ambassadors try to push technical discussions from their own local mailing lists or country-based mailing lists to regional lists that cover all the countries of Latin America, so that there are more opportunities for people to get the answers they need and also the mentorship they need to become future leaders. At this point, the conversation degraded a bit as people started to argue all the small details about whether to setup mail relays or encourage people gently to move to regional-based lists or to just shut down the smaller lists to encourage people to move to regional lists. Rodrigo Padula noted that the tools are already there to solve the problem — we just need to communicate! I shared a bit of my experience, and tried to get people back on talking about the principles and not getting bogged down in the details.

The next discussion revolved around events such as FUDCon. Guillermo noted that we’re pretty good at throwing events and inviting our friends to the party, but that we need to do more than that. He emphasized the fact that events are fun (like a party!), but we really need to focus on ensuring that we’re inviting people to become contributors. He said that there are several places people can get started in contributing — documentation, translation, packaging, infrastructure, marketing, design, and so on. We need to focus better on turning attendees into the folks that are going to be giving presentation in the next event. In short, events are the perfect location to invite people to work in Fedora.

The discussion then turned to ambassadors and the mentorship process, and whether or not the mentorship process is working as well as it should. Various ideas were tossed about, and I recommended that people make their suggestions to FAMSCo and the Fedora Board if they wanted to change things. (Several people suggested that we might need at least one or two more mentors here in LATAM.) From there, the conversation turned to the details of being a mentor (are you a coach or a filter?) and whether or not the 30-day probationary period for ambassadors is working or not. (Apparently it’s not in Brazil.)

To wrap things up, I thanked all the ambassadors and organizers for their hard work and effort to make this FUDCon a success, and encouraged them to look for opportunities during the rest of FUDCon and over the coming months to help new users understand that they’re going to have more fun if they are active participants in the community. I shared what I thought were the four most important things on my radar:

improved communication
making sure we start planning now for the next FUDCon in LATAM
making sure we include ambassadors and active contributors from many different countries, and not just send the same few people to each event
suggestion that we do a barcamp for at least a half-day in the next FUDCon in LATAM, so that people can see how it works and learn to do it better

By the time the round table wrapped up, it was late and I was exhausted, but people wanted to go out to eat, so I tagged along. To make a long story short, we found a good restaurant and spent the next several hours eating and talking. By the time we got back to the hotel, I was ready to crash, so I put off writing this blog entry until this morning.

Samsung Galaxy S (Vibrant on T-Mobile): First Day

2010-07-17T03:01:36+00:00

So technically yesterday was my first with the new phone, but it was late in the evening and I decided instead of try and formulate a review I would just get it setup and then use it for a day.

The phone is a dream! The 1GHz processor screams, and makes light work of heavy graphics and surfing. I am unfortunately stuck in an Edge network (Panama City, Florida) and so things are quite as snappy as they could be if I were back in SLC. All that aside things happen much faster still than they did with my old G1.

The Super AMOLED screen is dynamite! This is the best display I have ever seen on a phone (still haven't seen a Retina display). The images are crisp and the colors are deep and rich. It is truly an amazing screen.

The internal memory is a big treat too. I have currently 14GB free with 1.4GB free on the external memory card. This is quite welcome as I was able to move all things from my phone over and keep going as if I never moved.

Running Android 2.1 is nice, but I had that on my G1, so nothing new there. I have seen and heard rumors that 2.2 is coming for the Galaxy phones, but we will have to wait and see. Rooting the phone was a cinch and I won't look back. I can't wait to find a good mod for the phone (hopefully Cyanogenmod).

The best thing I can report about the phone is the work that Samsung put into battery life. Android phones have been notorious battery monsters. The G1 on a good day would go 5 hours without a charge. These days are gone, and I brought proof. Today I charged my phone to 100% capacity, then I took the phone off the charger for the whole day. With the exception of 1 reboot (which fubbed the numbers to be shorter) here are the results:

As you can see from the screen shot I went over 11 hours and 27 minutes off the plug. This is just amazing! It wasn't until the last hour or so that I noticed any real problems, like the camera not allowing pictures with such low battery.

The biggest draw back I have seen in the phone is that some features are turned off when on the phone or doing various tasks. For example you can't take a picture while talking on the phone... I don't know why.

The 5MP camera is nice, and quick to respond, would be nice to see a large MP but that is ok too.

Overall I think the Vibrant is a clear winner in the best Android phone category. Good work Samsung and kudos to Tmobile for picking it up.

FUDCon Santiago, Day 1

2010-07-16T17:32:58+00:00

Things are well under way here in Santiago, Chile at the FUDCon conference. Yesterday morning, my flight was delayed getting into Santiago, so when I landed I made a mad dash through immigrations and customs. Antonio was kind enough to pick me up at the airport and drive me directly to the conference. There is a nice new road from the airport to downtown, so it didn’t take very long to get to Universidad Central where the FUDCon conference is being held. When I arrived, the attendees were waiting for me to kick off the show. (I had secretly hoped they would start without me, but I didn’t get my wish.)

My plan was to give a quick introduction in (broken) Spanish, and then give a quick presentation in English. As it turned out, I ended up giving both the introduction and presentation in rusty Spanish, and even did a short question and answer session in Spanish as well. Little by little my brain is recalling words and phrases that I haven’t used in fourteen years.

After my presentation, I spent a few minutes meeting the other FUDCon organizers and staff, and then had some photos taken for the press here in Chile. (I’m not sure how well the pictures turned out, as I had just been on an airplane for almost ten hours!) I also had a telephone interview with a local journalist, which was a bit difficult because I found myself lacking in technical terms in Spanish. Luckily, I had Alexis and Loreto from the Universidad Central to help me better explain some of the more technical details in better Spanish.

After a quick lunch, I listened to a few of the other talks before being pulled out for more interviews and photos. It was very interesting to listen to the questions from the journalists, and see how they started to catch on to the “open source way”. Alexis also shared with them more statistics about software piracy here in Chile (and Latin America in general), and how free and open source software is a natural way to help solve that problem.

After the conference wound down for the night, we returned to the hotel to drop off our bags and then went out for some dinner. We ended up eating at a restaurant with some great Peruvian food. After a long and exciting day, I slept like a rock.

Car Update

2010-07-16T14:30:55+00:00

For those that had been interested I have traded my 1964 Bug. The kid that has it seems like he will treat it really well and I am glad that he got it. As I was getting it ready for him I drove it one more time and just thought about how fun of a car it really was to drive and have. All the looks that I got and people wanting to know about that classic car.

But I digress. I traded it for his 1988 Chevy pickup. I am currently getting the title but then I am going to put that up for sale. If it doesn’t go fairly quick I will take it down to Carmax and see what they do.

So two questions, first does anybody want a pickup truck? If so message me or comment and I can get you the info. Second, does anyone have experience selling a car to car max? Good or bad I am curious to hear it.

PHP Fundamentals

2010-07-16T00:06:19+00:00

For our July meeting, Justin Carmony will continue his PHP Fundamentals series. Come join us for valuable training, good times, and of course, the killer after party.

Using ANTLR and PerlXS to Generate a Parser

2010-07-15T19:10:10+00:00

As I mentioned earlier, we're anticipating changing out the current Parse::RecDescent based parser in the Kynetx platform with one that will perform better. We've been going down the path of using ANTLR, a modern parser generator that supports multiple target languages. That flexibility was one of the key thing that got us interested in ANTLR. We might want to generate Ruby or Javascript KRL generators at some point.

But of course right now we want to generate a Perl parser since that's what the underlying Kynetx Event Service (KES) is written in (it's an Apache module). ANTLR doesn't support Perl. That's probably just as well however since we're after as much speed as possible. We could generate Java (the target best supported by ANTLR) but adding Java servers into the current operational mix doesn't excite me.

The obvious course is to generate C and then use PerlXS to integrate the resulting parser into the Perl-based KES engine. To explore the feasibility of that, I decided to play around with ANTLR generated C parsers and PerlXS to see how they'd work. What follows is an intermediate report of what I found.

I started with a SimpleCalc example that is part of the five minute introduction to ANTLR. The grammar file is unchanged from that example:

grammar SimpleCalc;

options
{
    language=C;
    output=AST;
    ASTLabelType=pANTLR3_BASE_TREE;

}

tokens {
	PLUS 	= '+' ;
	MINUS	= '-' ;
	MULT	= '*' ;
	DIV	= '/' ;
}


/* PARSER RULES */
expr	: term ( ( PLUS | MINUS )^  term )*;
term	: factor ( ( MULT | DIV )^ factor )* ;
factor	: NUMBER ;


/* LEXER RULES */
NUMBER	: (DIGIT)+ ;
WHITESPACE : ( '\t' | ' ' | '\r' | '\n'| '\u000C' )+ 	
     { $channel = HIDDEN; } ;
fragment DIGIT	: '0'..'9' ;

The only difference is that I've told it to generate an AST and annotated the grammar (with ^) to tell it which tokens are tree nodes.

I used h2xs to generate the boilerplate xs files:

h2xs -A -n SC

This creates a directory called SC and a punch of files for PerlXS. That's where I put all the generated files from ANTLR. If you look at the C version in the ANTLR introduction, you'll see an @members declaration that contains some C code that exercises the parser. That's what I modified to put into the PerlXS file:

#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"
#include "SimpleCalcLexer.h"
#include "SimpleCalcParser.h"

#include "ppport.h"

MODULE = SC		PACKAGE = SC		

char *
showtree(in)
         char * in
    CODE:

    pANTLR3_INPUT_STREAM           input;
    pSimpleCalcLexer               lex;
    pANTLR3_COMMON_TOKEN_STREAM    tokens;
    pSimpleCalcParser              parser;
    SimpleCalcParser_expr_return     langAST;

    char * output;

    input = antlr3NewAsciiStringInPlaceStream 
            (in,
            (ANTLR3_UINT32) strlen(in), 
            NULL); 
    lex    = SimpleCalcLexerNew(input);
    tokens = antlr3CommonTokenStreamSourceNew
               (ANTLR3_SIZE_HINT, 
		TOKENSOURCE(lex));
    parser = SimpleCalcParserNew(tokens);

    langAST = parser->expr(parser);

    output = langAST.tree->toStringTree(langAST.tree)->chars;

    // Must manually clean up
    //
    parser ->free(parser);
    tokens ->free(tokens);
    lex    ->free(lex);

    RETVAL = output;

    OUTPUT: 
      RETVAL

This defines a function called showtree that will be called from Perl. The file also includes the .h files that ANTLR generated and uses the input string inplace instead of reading a file as the example did. The return value (denoted by the special identifier RETVAL) is just a string representation of the parse tree.

The Makefile.PL file for PerlXS is pretty standard:

use 5.010000;
use ExtUtils::MakeMaker;
WriteMakefile(
    NAME              => 'SC',
    VERSION_FROM      => 'lib/SC.pm', 
    PREREQ_PM         => {}, 
    ($] >= 5.005 ?     
      (ABSTRACT_FROM  => 'lib/SC.pm', 
       AUTHOR         => 'Web-san ') : ()),
    LIBS              => ['-lantlr3c'], 
    DEFINE            => '', 
    INC               => '-I.', 
    # link all the C files too
    OBJECT            => '$(O_FILES)',
);

You'll notice I link the ANTLR library in here. Since the xs file references a string output, I created a typemap file to map that for PerlXS:

TYPEMAP
char * T_PV

Now, we compile the xs files in the standard way:

perl Makefile.PL
make

The Perl file is pretty simple as well:

#!/usr/bin/perl -w

use ExtUtils::testlib;   # adds blib/* directories to @INC
use SC;
print SC::showtree("3 + 4 * 5"), "\n";

Executing this program prints a prefix representation of the arithmetic expression passed into the showtree function.

Of course, this isn't what we want for our system. We want a full fledged AST back that we can manipulate in Perl. I spent a little time on typemaps and have reached the conclusion that the right method is to use an ANTLR generated treeparser (a parser for the AST) to walk the tree and create a tree that is more like what we are used to in the KES engine and use typemap to turn that into Perl.

So, it would appear that using ANTLR to generate a C-based parser and then using PerlXS to wrap that for use in Perl is feasible. As we figure out the AST output, I'll write more.

Tags: perl perlxs antlr parsing kynetx krl

Last day to submit paper for UTOSC 2010

2010-07-15T18:32:16+00:00

Today is the last day to submit a paper for the Utah Open Source Conference 2010.
Delivering a paper is a GREAT way for you to give back to the community and also for you to network and get to meet new people.

You don’t have to have a doctorate to present, so post your idea today! More info below

mj/v

ps. submitting a proposal for a presentation doesn’t have to be a painful reminder of a college paper or a dissertation, write your ideas in english, spanish, chinese pinyin, or uuencode!
The Utah Open Source Foundation announced a call for papers for the
2010 conference scheduled Oct 7 – 9 in Salt Lake City, Utah. July 15th
is the last day to submit proposal for review is July 15th and we
look forward to hearing your suggestions for papers.

All papers related to open source and open standards technology are
welcome including presentations on software best practices, workshops,
panel discussions, etc. Topics can range in sophistication from
beginner level classes to advanced training. We also make space
available for local user groups to meet and find new members,
participate in Birds of a Feather sessions and educational
minicourses.

With open source adoption showing brisk growth in the Mountain West,
UTOSC expects another record-breaking year of attendance, making this
conference a great place to discover, explore and learn new
technologies and software.

Please visit http://utosc.com/ for more details.

Android 2.2

2010-07-15T14:30:34+00:00

I recently installed Android 2.2 on my phone. Gotta thank the guys at cyanogenmod for the awesome ROM that they put out. Of all the ROMs I have tried they are by far the most polished and nice. Plus the ease of using ROMManager is a no brainer. So my thoughts on Android 2.2.

There are some great improvements. First, I thought that I would like having 360 degree screen rotation more. But the reality is that I move my phone two ways. After using everything for a bit I find the rotation to be more of a ‘see I can do this’ thing rather than a useful feature. But still, see I can do this!

More notable for me is the improvements in both the Gmail app, and the Calendar app. Two things in the Gmail app. The support for the color labels now. If you haven’t used this it does provide a great ‘at a glance’ showing of what is going on with your labels. Also the app UI is improved, with some buttons added for fast account switching for those of us with multiple gmail accounts, and more handy next and forward buttons. Also, just found the option now to ‘select text’. When selected you can highlight message text and when done it will auto copy to the clipboard.

The calendar app has a much needed improvement of being able to sync with multiple gmail accounts. I can’t even say how frustrating this was. I had so many people that signed up for a new gmail account, and when we added their work gmail account they couldn’t have the work calendar.

Other improvements seem to mostly be UI improvements with things like the Talk application. Changes in the ‘Car Home’ UI. Call log now has groupings so if you miss 80 calls from your boss in a row you just see one entry with a ‘(80)’ next to it. This will of course show all 80 once you expand them. Contacts now also has a quick dial button at the full list so that you can call directly rather than long pressing or going into the contact. The ‘Messaging’ app has added contacts pictures in the view messages pane and, has changed the font on the date and time stamp so you can actually read it without your microscope. Camera UI has also improved giving quick access to features such as flash, white balance, GPS location, etc. Also who can forget flash support with the betas from Adobe. Lastly, the market app has some cleaning up making it easier to leave feedback, and a dedicated tab to view comments.

Although I haven’t noticed any great speed changes all in all it is very worth the upgrade.

Some things I would still love to see in Android. Universal Copy and Paste. I mean, come on. I can copy a SMS text but not a talk chat? Why not just be able to highlight any text and copy it, paste it, etc. While the ‘select text’ option in gmail is a step in the right direction I don’t think it is enough. It would also be nice to see universal screen rotation. Some apps just won’t rotate unless I pull out the keyboard. Also, a native buzz app would be cool. Until then I will have to keep using Seesmic. Last but not least some improvements to the google voice app. Specifically ability to record calls that are initiated from the phone running the app, and multi-select for bulk actions on things like deleting and archiving.

Travel delays and broken feeds

2010-07-14T15:09:10+00:00

Due to the weather in the Atlanta area yesterday, my flight (connecting through Atlanta) to FUDCon Chile was cancelled yesterday. This means I’ll be arriving at FUDCon Santiago a day later than planned — I arrive early Thursday morning instead of Wednesday Hopefully this won’t cause too much disruption. Unfortunately, it will give me one less day meet people and help with arrangements for the conference.

In other news, I’ve hacked on the WordPress 3.0 code used to power this blog, and we’ll see if it fixes the “truncated posts in RSS feeds” bug I’ve been having. This post should work as a test to see if my changes have worked. With any luck, this’ll fix the problem and I can spend less time working on the code and more time blogging!

Geek Squad Experience

2010-07-14T14:30:07+00:00

I think we have all seen the little Volkswagon Bugs plastered with their oval logo. Or their vans, all decked out in black, white, and orange. If you have ever been to BestBuy you can’t even purchase a DVD player without the offer of a Geek Squad ‘Double Agent’ to come and install the item. They offer in home diagnosis of computer problems and can do computer repairs.

I have heard of others having poor experiences with Geek Squad. My brother Ryan had a computer gone for 12 weeks. Each day a new excuse as to why it wasn’t done. The other day one of the computers at our floral show was on the fritz. In fact, two were. I went to look at the computers but was only able to dedicate a limited time to my diagnosis. Not being able to complete the repair the choice was made to call Geek Squad. They needed to have the computer fixed and it made sense to have someone come over that was a ‘computer expert’ and would make ‘house calls’.

Geek Squad arrived and went to work addressing the problems. First, was a computer that was being plagued by pop-ups. Whenever I went to diagnose the problem no pop-ups happened and I didn’t see anything out of the ordinary in the processes list. Geek Squad looked at the computer and said that the hard drive had bad sectors and the computer needed to be replaced. Well, that may be true, but bad sectors don’t cause pop-ups. They found the first thing wrong and the first suggestion was to replace the computer. Not even just replace the hard drive, the whole computer needed to be new.

They then spent a few hours looking at the next computer. This computer needs to use the modem, and what was happening was that the modem would work for a short time, and then stop working. When you would try to make a call, it would say the telephony service had an error, and restarting the telephony service didn’t help. Geek Squad still hasn’t figured it out. I am waiting for them to say that we need a new computer or it will take 12 weeks in their shop.

The point is they never did address the problems that we asked them to. It seems they just want to ‘up-sell’ the consumer. In addition their diagnosis cost was enough that I would have expected more. Has anyone had a good Geek Squad experience?

Parsing for a Cloud-Based Programming Language

2010-07-13T22:25:03+00:00

If you follow my blog, you're probably all too aware that Kynetx provides a cloud-based execution engine for a programming language called the Kynetx Rule Language, or KRL. The execution engine is a big, programmable event-loop in the cloud and KRL is the language that is used to program it. As with any language, the first step in executing it is to parse it into an abstract syntax tree.

The KRL parser has had it's ups and downs. A while ago, I reported on changing out the HOP parser for Damian Conway's RecDescent (note that the system is written in Perl). The KRL parser is also what promted me, a long time ago, to write a large set of tests for the engine code, including the parser.

When you're building a parser for a language to be compiled on the developer's machine, performance is an issue to be sure, but you are saved from problems you encounter when the parser is the first step in a service that needs to run quick. The most important of these is to make sure you cache the abstract syntax tree so that you only have to parse things when they change.

Whenever there's a cache miss the KRL parsing process basically works like this:

Retrieve the KRL source for the ruleset from the rule repository.
Parse the ruleset
Optimize the ruleset
Compile event expressions into state machines
Cache the result

The optimization does things like perform dependency analysis on variables to move them outside loops and so on.

The caching presents it's own problems. Flushing a single ruleset from the cache when it changes is a simple enough procedure. But when the abstract syntax tree (AST) changes, they all need to be flushed. My preference is to do this in an emergent way so that we're not dependent on a step in the code deployment process that might get forgotten or messed up. So, when a ruleset is optimized, an AST version number is placed in the AST. When the engine retrieves a ruleset from the cache, it checks the AST version number against the current number and if it's old, automatically flushes the ruleset and proceeds with the process given above to recreate the AST.

So far, so good. When I make a change to the parser that would change the format of the AST, I update the version number and when the code is deployed, the rulesets will automatically flush and get regenerated. This is, unfortunately, where my intuition failed me.

You can imagine that the engine is a number of individual servers behind a load balancer. When multiple requests come in they are given to various server to process. Like most people, I've been conditioned by the physical world to expect guassian distributions around an average, but intuition developed around guassian distributions often leads to poor decisions in the world of computers where distributions are much more likely to follow a power curve.

Here's what happened the last time I made a change to the AST. We deployed code and immediately the system stopped responding to requests. On further examination, we discovered that the machines we all busy parsing--the same ruleset. Because there are some rulesets that are much more likely to be executed than others, the first requests seen by the new code were, for the most part, all for the same ruleset and every machine was busy parsing. Worse, the requests started to stack up cause the machines to thrash. Everything stopped working.

The answer to this problem has two parts:

The parser needs to be faster--especially for large rulesets. In the short term we'll do some point imporvements to the existing Perl-based parser. In the long term we'll replace it with a C-based parser.
The rule retieval process detailed above needs to use semaphores so that only one parsing task for any given ruleset is launched at one time.

We've put the short term fixes and the semaphore in place and things are looking pretty good so far. We're partly down this road already and the fact that we have good specifications of what the parser has to do and tests to check it out thoroughly, I'm not worried about the change.

I used the cache as an interprocess commmunication system to create the semaphores. It's not strickly speaking atomic, but it's close enough and was dirt simple to implement.

Tags: krl kynetx parsing

Monthly presentations now at Code Greene in Salt Lake

2010-07-13T18:34:16+00:00

As you’ve probably recently heard, our great and fearless leader Victor ‘mindjuju’ Villa has made some changes, and our previous venue will no longer be available, on top of which he has passed his UPHPU baton to me.

While it is definitely not a requirement that the meetings be held at the place of employment of the current president, that tradition will continue for the foreseeable future, as he new meeting location is my place of employment. We’re the one with the big green stripe on the front window and door. Note that you can get to us via southbound State Street at 350 S onto Exchange Place, or via westbound 400 South at about 30 E, onto Cactus Street. Many of you may be familiar with XMission, and we’re right across the back alley from them. Also important to note is that parking on the street at meters is free after 6pm, and parking in the garage across the street will cost $5 for the evening, and we will be unable to validate. We will continue to meet at seven o’clock on the evening of the third Thursday of each month.

There is an Applebee’s at the Gateway, a short free Trax ride away from our meeting location, so I anticipate we’ll head over there en masse after the presentation for the after-party. They also validate an hour of parking I think if anyone wanted to drive over there rather than come back to get a car.

If we find that the downtown location is for whatever reason undesirable, we can switch things up. Since I live in Draper, I would be happy to do the after party at the Draper Applebee’s if people would prefer, but I suspect our friends from the North might not like driving down there just for a different Applebee’s, but I could be wrong.

First day on the job

2010-07-13T03:26:56+00:00

My T510 laptop

As many of you may well know, today was my first day as a new employee of Red Hat and as the new Fedora Project Leader. I spent the day today in “new hire orientation” meetings at Raleigh, which were interesting. I was impressed by how much time Red Hat took to make sure the new recruits understood the open source way, and where Fedora fits in the mix of things. Some of the highlights for me were Max Spevack explaining open source communities and their importance, and Michael Tiemann (one of my favorite speakers to listen to) giving an excellent presentation on how open source can be a transforming tool for information technology departments. We also spent time filling out paperwork and making sure we had all our ducks in a row, and ended the day with an ice-cream social. I’ll be in new hire orientation tomorrow as well, and then head directly to FUDCon Santiago tomorrow evening. Paul Frields has graciously agreed to help me get up to speed in my new role, which I especially appreciate since I’ll spend the next two weeks on the road.

This evening I also was able to start getting Fedora installed on my new laptop. I opted for a Lenovo T510, mostly because it had been years since I’d used a new ThinkPad, and I wanted to try something that was likely to be a bit outside of the mainstream. (I have this weird habit of finding the oddball hardware bugs in Fedora. Some people go out of their way to make sure their hardware is going to be compatible. As for me, I throw Fedora on weird hardware and see what breaks. Call me crazy…) This laptop is a bit bigger than most people would want to carry around, but I’m happy with it.

Paul Frields was kind enough to take a first stab at getting the laptop setup while I was still in my meetings. Apparently there’s an issue with the new VT-d feature (Virtualization Technology for Directed I/O) of the Intel chipsets, as Paul had to disable it in the BIOS to keep anaconda from crashing with what appeared to be memory-corruption issues. Once we got that tracked down and disabled in the BIOS, I threw Fedora on the machine and it’s been running great ever since. So far, everything has worked great. I was a little worried about how well this new wirless chip would be supported, but it’s working very well. I was even able to setup a quick ad-hoc wireless network for Paul to be able to connect his laptop through mine. Suspend/resume worked out of the box, as did the Nouveau driver. There’s a bit of funkiness with the reporting of the second battery, but I’ll do some more troubleshooting on that over the next few days and file a bug.

The day ended with an excellent sushi dinner (thanks Paul!), and now I’m back at the hotel getting more software installed on the laptop and working on presentations for FUDCon Chile and FISL. All in all, I can’t complain about my first day on the job!

black raspberry bounty

2010-07-11T16:47:00+00:00

year 2 for the berries, and they are rocking. this am we got 37 more for a total of 98 so far.

Published with Blogger-droid v1.4.5

Install Adobe Reader 9.3.2 on Ubuntu 10.04 LTS “Lucid Lynx”

2010-07-11T15:44:42+00:00

The topic of installing Adobe products in Ubuntu has both been loved and hated by reader since the first time I outlined it. Some very much appreciate the clear steps outlining installation of the PDF reader application. Others despise the idea of proprietary software intermingling with their otherwise open source desktop. However you feel about Adobe Reader, love it or hate it, it is a popular application and in many cases required by users for work or otherwise. With each new Ubuntu release I outline how to install and update your Adobe Reader installation.

Repository Requirements (Optional)

Canonical, the parent company behind Ubuntu, has provided a repository to distribute Adobe Reader and similar applications. The method of installing Ubuntu by way of a repository is more automated than a direct download from the Adobe website. I’ll outline either version, but keep in mind that I prefer (and suggest!) the first. Configuring the partner repository is a requirement for the first option.

Canonical Partner Repository – Option 1

Installation of Adobe Reader 9 requires the activation of the Canonical Partner repository. You can add the Partner repository by following the steps outlined below.

Navigate to System > Administration > Software Sources
Select “Third Party” or “Other Software”
Select “Add” and enter: deb http://archive.canonical.com/ lucid partner

You should now be able to install the latest version of Adobe Reader on Ubuntu 10.04 LTS by using the following command, or clicking the embedded link:

sudo aptitude install acroread

Direct Download: Adobe.com – Option 2

The alternate installation solution is to download Adobe Reader directly from the Adobe website. As a disclaimer, I should warn you that this method will not receive automatic updates and it is left to the user to download and install any future releases of this application.

If you understand these requirements, continue to install Adobe Reader using the following steps:

Visit this page: http://get.adobe.com/reader/otherversions/
Select “Linux – x86 (.deb)” from the drop-down menu
Select your preferred language
Download the package

Depending on your browser you may be prompted to open the package with the package installer, or it will simply save it to disk. If it saves to disk, you’ll simply need to double-click the archive for the installation to begin.

Conclusion

Most of my dealings with PDF files is managed by Evince, the default GNOME pdf reader. Occasionally however I need the Adobe-specific application. As with many of the readers here, I prefer the free software solution where possible, but sometimes it just isn’t possible. Love it or hate it, Adobe Reader is sometimes part of our lives.

Again, I would suggest the first solution toward installing Adobe Reader as it will automatically manage security updates for you. The packages are provided directly from Canonical, Ubuntu’s parent company, and managed by Ubuntu staff. I trust these packages just as much (if not more) than the direct-from-adobe packages available in the secondary solution.

January 14, 2010 — Install Adobe PDF Reader 9 on Ubuntu 9.10 “Karmic Koala” (9)
June 23, 2008 — Install Adobe Acrobat Reader 8.1.2 on Ubuntu 8.04 (74)
May 6, 2010 — Ubuntu 10.04 “Lucid” Blank Screen at Startup : Workaround (57)
May 4, 2010 — Upgraded to Ubuntu 10.04 “Lucid Lynx” (7)
January 15, 2010 — Ubuntu 10.04 “Lucid Lynx” Alpha 2 Released (2)
September 19, 2009 — Lucid Lynx? Ubuntu 10.04 Codename Announced (4)
July 3, 2008 — Print Directly To PDF in Ubuntu 8.04 (18)
June 25, 2008 — Install Adobe Acrobat Plugins For Firefox (8)
January 29, 2008 — Medibuntu : The Only 3rd Party Repo I Use (11)
December 26, 2007 — Manually install Adobe Flash / Temporary flashplugin-nonfree fix (18)

Google map with Utah Repeaters

2010-07-10T17:08:15+00:00

I thought it would be helpful to visualize where all the radio repeaters are in my area. I found a lack of map information available for this sort of thing but I found readily available CSV data with the information. Google Fusion Tables to the rescue! I tried to embed the map in this post [...]