Top 10 Software Engineering Books – Must Read

Today I gave a lighting talk at SEITENBAU, about my favorite top ten software engineering books. Here my top ten list of books, which every developer should read:

#1 - Andrew Hunt, David Thomas - The Pragmatic Programmer - From Journeyman to Master

#2 - Erich Gamma, Richard Helm, Ralph E. Johnson, John Vlissides - Design Patterns

#3 - Martin Fowler - Refactoring -Improving the Design of Existing Code

#4 - Eric Evans - Domain Driven Design

#5 - Kent Beck - Test Driven Development by Example

#6 - Alistair Cockburn - Writing Effective Use Cases

#7 - Scott Berkun - The Art of Project Management

#8 - Martin Fowler - Patterns of Enterprise Application Architecture

#9 - Joshua Bloch - Effective Java

#10 - Gerard Meszaros - xUnit Test Patterns

I think there are more books you should read as a professional developer, but this list is at the moment my top ten list of books I think should have read.

Top10- Software Engineering Books

View more presentations from Christian Baranowski

What are your favorite software engineering books? Send me comments or write me via Twitter.


Links

• Stackoverflow recommended books for software engineering- http://stackoverflow.com/questions/131571/recommended-books-for-software-engineering
• Top 100 Best Software Engineering Books- http://knol.google.com/k/top-100-best-software-engineering-books-ever

Sikuli – GUI Test Automation with Java Robot API and Images

The Sikuli project provides a simple tool for automate and test graphical user interfaces (GUI) using images (screenshots). The idea is to find an element on the screen by a screenshot and not by XPath or ID. The tool can be used for all GUIs also web applications. It’s a nice idea below you can see a simple example.

The SIKULI project is based on the Java VM and also provides an API for writing the automation in Java here a JUnit sample test.

	import org.junit.Test;
	import org.sikuli.script.App;
	import org.sikuli.script.Screen;
	public class GoogleDemoTest {
	@Test
	public void testGoogleSearch() throws Exception {
	Screen screen = new Screen();
	App app = new App("Firefox 4.app");
	app.focus();
	screen.click("images/homeButton.png", 0);
	screen.click("images/homeUrlInput.png",0);
	screen.type(null, "www.google.com", 0);
	screen.click("images/goButton.png", 0);
	screen.click("images/searchInput.png", 0);
	screen.type(null, "tux2323", 0);
	screen.click("images/searchButton.png", 0);
	screen.click("images/tux2323BlogLink.png", 0);
	}
	}

view raw GoogleDemoTest.java hosted with ❤ by GitHub

For real world web testing it is not the right tool I think, because it depends on the style of the elements. When the button style changed you need a new screenshot of the button. But it is a nice tool for simple automation tasks. Have also a look at screen casts on the project homepage.

Links:
- Project Home - http://sikuli.org/
- How to use Sikuli Script in your JAVA programs - http://sikuli.org/docx/faq/030-java-dev.html

Convert a simple CSV File with Groovy, Java, Scala, awk, Ruby, Python, PHP or Bash?

Change Log:

• 05.06.2011 1:30 pm - Initial created Post with  Groovy, Java, Scala, awk, Ruby, Python Implementation
• 05.06.2011 4:00 pm - Add PHP implementation and update voting (now you can vote for PHP).
• 06.06.2011 4:00 pm - Add Bash implementation and update voting (now you can also vote for Bash)

Which is the best programming language for converting a simple CSV into another format?

First I blogged three Java VM based solutions written in Groovy, Java and Scala to convert a simple CSV file into another format. Rainer sends me the Java based solution, yesterday Axel Knauf sends an awk based solution, Niko sends Ruby based solution, Hendrik sends a Python based solution, Sebastian sends me a PHP implementation and Julien sends a Bash version. Now there are a Groovy, Java, Scala, awk, Ruby, Python, PHP and Bash implementation.

Now here again a complete overview of the different implementations:

The Groovy Implementation:

	new File("output.csv") withPrintWriter { out ->
	new File("input.csv") splitEachLine(';') { fields ->
	def name = fields[2]
	def firstname = fields[1]
	def kto = fields[3]
	def blz = fields[4]
	def amount = fields[5]
	out.println "${name};${firstname} ${name};${kto};${blz};${amount}"
	}
	}

view raw csvconvertor.groovy hosted with ❤ by GitHub

The Java Implementation:

	import java.io.*;
	public class CsvConvertor {
	public static void main(String[] args) throws Exception {
	FileWriter out = new FileWriter("output.csv");
	BufferedReader in = new BufferedReader(new FileReader("input.csv"));
	String line;
	while ((line = in.readLine()) != null) {
	String[] fields = line.split(";");
	String name = fields[2];
	String firstname = fields[1];
	String kto = fields[3];
	String blz = fields[4];
	String amount = fields[5];
	out.append(
	String.format("%s;%s %1$s;%s;%s;%s%n",
	name, firstname,kto, blz, amount));
	}
	out.close();
	}
	}

view raw CsvConvertor.java hosted with ❤ by GitHub

The Scala Implementation:

	import io.Source._
	import java.io._
	object CsvConvertor extends Application {
	val outputCsv = new FileWriter("output.csv")
	val accounts = fromFile("input.csv") getLines() map (line => Account(line))
	accounts foreach (account => outputCsv append (account toCsv))
	outputCsv close
	}
	case class Account(line: String) {
	val data = line split (';')
	val firstname = data(1)
	val lastname = data(2)
	val kto = data(3)
	val blz = data(4)
	val amount = data(5)
	def toCsv() =
	"%s;%s %1$s;%s;%s;%s%n" format (lastname, firstname, kto, blz, amount)
	}

view raw CsvConvertor.scala hosted with ❤ by GitHub

Here the shell command and awk script:

awk 'BEGIN { FS=";"; OFS=";" } { print $2,$1" "$2,$3,$4,$5 }' input.csv > output.csv

view raw convert.awk hosted with ❤ by GitHub

The pure Ruby Implementation:

The Python Implementation:

	import csv
	with open('input.csv', 'rb') as input, open('output.csv', 'wb') as output:
	reader = csv.reader(input, delimiter=';')
	writer = csv.writer(output, delimiter=';')
	for input_row in reader:
	firstname, name, accno, bsc, amount = input_row
	output_row = [name, '%s %s' % (firstname, name), accno, bsc, amount]
	writer.writerow(output_row)
	# For Jython 2.5, the context manager usage ('with') has to be replaced with a classic try..finally.
	# For Python 3.x, the files have to be opened in text mode ('r', 'w') instead of binary mode ('rb', 'wb').

view raw CsvConverter.py hosted with ❤ by GitHub

The pure PHP Implementation:

	<?php
	$start = microtime(true);
	$fpIn = fopen('input.csv', 'r');
	$fpOut = fopen('output-pure.csv', 'w');
	while (($row = fgets($fpIn)) !== false)
	{
	$fields = explode(";", $row);
	fwrite($fpOut,
	implode(
	";",
	array(
	"name" => $fields[0],
	"firstname name" => '"'.$fields[1] ." ". $fields[0].'"',
	"kto" => $fields[2],
	"blz" => $fields[3],
	"amount" => $fields[4]
	)
	)
	);
	}
	$end = microtime(true);
	$duration = $end - $start;
	echo "Duration: ".round($duration, 2) . "s".PHP_EOL;

view raw csv-format-pure.php hosted with ❤ by GitHub

The Bash Implementation:

	#!/bin/bash
	while IFS=$';' read -r name vorname kto blz amount
	do
	echo "$name;$vorname $name;$kto;$blz;$amount" >> output.csv
	done < input.csv

view raw converter.sh hosted with ❤ by GitHub

I'm curious whether there are other implementation proposals (Clojure, Perl, PHP, …), if you have one you could send me the script via Twitter or leave a comment here…

I am also curious which implementation Groovy, Java, Scala, awk or Ruby you like and why? I have create voting here:

Thanks Rainer, Axel Knauf, Niko Dittmann, Hendrik Heimbuerger S.Barthenheier and Julien Guitton for the Java, awk, Ruby Python, PHP and Bash implementation.

Links:

• Java (Rainer) - https://gist.github.com/1006757
• awk (Axel Knauf) - https://twitter.com/#!/kopfkind/status/77288701725638656
• Ruby (Niko) - https://gist.github.com/1008877
• Python (Hendrik) - https://gist.github.com/1008881
• PHP (Sebastian) - https://gist.github.com/1008966
• Bash (Julien) - https://gist.github.com/1009420
• old blog post: Convert a CSV File with awk - http://goo.gl/U7onj
• old blog post: Convert a CSV File in Groovy, Java or Scala? - http://goo.gl/AAE17

Convert a CSV File with awk

Yesterday I post three Java VM based solutions written in Groovy, Java and Scala to convert a simple CSV file into another format. Today kopfkind sends me via Twitter a awk based solution.

Here the shell command and awk script to convert the CSV file:

awk 'BEGIN { FS=";"; OFS=";" } { print $2,$1" "$2,$3,$4,$5 }' input.csv > output.csv

view raw convert.awk hosted with ❤ by GitHub

Thanks a lot @kopfkind for this simple solution. I'm curious whether there are other implementation proposals (Python, Clojure, Perl, Bash, PHP, …), if you have one you could send me the solution via Twitter or leave a comment here.

I am also curious which implementation Groovy, Java, Scala or awk you like and why?

See also: Convert a CSV File in Groovy, Java or Scala?

Convert a CSV File in Groovy, Java or Scala?

Last week I have simple task I must convert a simple CSV file into another CSV format. My first solution was a simple Groovy script. Then inforw sends me a Java solution, to show me that with Java it is no much more code then the Groovy implementation is. Today I wrote just for fun a solution in Scala, to see how the code looks in Scala. My favorite of the three implementations is at the moment the Groovy one. But I think the Scala implementation has the best readability. Below you see the three implementations.

I'm curious what you like, feel free for comments? And I would be glad if someone contributes even further implementation in Clojure, Python, Perl,… or even a better Scala, Java or Groovy implementation.

The Groovy Implementation:

	new File("output.csv") withPrintWriter { out ->
	new File("input.csv") splitEachLine(';') { fields ->
	def name = fields[2]
	def firstname = fields[1]
	def kto = fields[3]
	def blz = fields[4]
	def amount = fields[5]
	out.println "${name};${firstname} ${name};${kto};${blz};${amount}"
	}
	}

view raw csvconvertor.groovy hosted with ❤ by GitHub

The Java Implementation:

	import java.io.*;
	public class CsvConvertor {
	public static void main(String[] args) throws Exception {
	FileWriter out = new FileWriter("output.csv");
	BufferedReader in = new BufferedReader(new FileReader("input.csv"));
	String line;
	while ((line = in.readLine()) != null) {
	String[] fields = line.split(";");
	String name = fields[2];
	String firstname = fields[1];
	String kto = fields[3];
	String blz = fields[4];
	String amount = fields[5];
	out.append(
	String.format("%s;%s %1$s;%s;%s;%s%n",
	name, firstname,kto, blz, amount));
	}
	out.close();
	}
	}

view raw CsvConvertor.java hosted with ❤ by GitHub

The Scala Implementation:

	import io.Source._
	import java.io._
	object CsvConvertor extends Application {
	val outputCsv = new FileWriter("output.csv")
	val accounts = fromFile("input.csv") getLines() map (line => Account(line))
	accounts foreach (account => outputCsv append (account toCsv))
	outputCsv close
	}
	case class Account(line: String) {
	val data = line split (';')
	val firstname = data(1)
	val lastname = data(2)
	val kto = data(3)
	val blz = data(4)
	val amount = data(5)
	def toCsv() =
	"%s;%s %1$s;%s;%s;%s%n" format (lastname, firstname, kto, blz, amount)
	}

view raw CsvConvertor.scala hosted with ❤ by GitHub

Thanks @inforw for the discussion and the Java implementation.

Test Logging via JUnit Rule

In most cases it is not necessary to test logging, but in some situations you wont to check in a JUnit test that a log statement will be written, e.g in special error cases. When you use logback as logging framework the test can setup a ListAppender for a Logger and can implement a assert method which checks that the expected log statement is in the ListAppender list. But then the test is logback specific and has dependency on logback! When you use this approache in many JUnit test classes the test source has a hard dependency on logback and when you wont to change the logging framework all tests must also be changed. To remove this hard dependency you can use a JUnit rule, which capsulate the logback framework from the test source. The example here shows how a test and such a JUnit rule can look like.

Here the sample test for verify logging:

	import static org.junit.Assert.*;
	import static org.hamcrest.Matchers.*;
	import java.util.Stack;
	import org.junit.Before;
	import org.junit.Rule;
	import org.junit.Test;
	import org.slf4j.Logger;
	import org.slf4j.LoggerFactory;
	// Sample Logging Test
	public class SimpleStackTest {
	@Rule public LogStub logStub = new LogStub(){{
	record(LogLevel.INFO);
	recordLoggingForType(SimpleStack.class);
	}};
	SimpleStack stack;
	@Before
	public void setup() {
	stack = new SimpleStack();
	}
	@Test
	public void testPush() {
	stack.push("Item One");
	assertTrue(logStub.containes("Push item Item One into stack."));
	assertThat(logStub.size(), is(1));
	}
	}
	//SUT
	class SimpleStack {
	private static final Logger logger = LoggerFactory
	.getLogger(SimpleStack.class);
	private Stack<String> stack = new Stack<String>();
	public void push(String item) {
	final String method = "push() :";
	logger.trace("{} Start", method);
	stack.push(item);
	logger.info("{} Push item {} into stack.",method, item);
	logger.trace("{} End", method);
	}
	}

view raw SimpleStackTest.java hosted with ❤ by GitHub

And here the logback specific rule:

	import java.util.List;
	import java.util.Vector;
	import org.junit.rules.TestWatchman;
	import org.junit.runners.model.FrameworkMethod;
	import org.slf4j.LoggerFactory;
	import ch.qos.logback.classic.Level;
	import ch.qos.logback.classic.Logger;
	import ch.qos.logback.classic.LoggerContext;
	import ch.qos.logback.classic.spi.ILoggingEvent;
	import ch.qos.logback.core.read.ListAppender;
	public class LogStub extends TestWatchman {
	public enum LogLevel {
	TRACE(Level.TRACE),
	DEBUG(Level.DEBUG),
	INFO(Level.INFO),
	WARN(Level.WARN),
	ERROR(Level.ERROR);
	Level internalLevel;
	private LogLevel(Level level) {
	this.internalLevel = level;
	}
	}
	private final ListAppender<ILoggingEvent> listAppender = new ListAppender<ILoggingEvent>();
	private final LoggerContext lc = (LoggerContext) LoggerFactory
	.getILoggerFactory();
	private final Vector<Class> loggingSources = new Vector<Class>();
	private LogLevel level = LogLevel.TRACE;
	@Override
	public void starting(FrameworkMethod method) {
	before();
	}
	@Override
	public void finished(FrameworkMethod method) {
	after();
	}
	public void before() {
	resetLoggingContext();
	for (Class logSource : loggingSources) {
	addAppenderToType(logSource);
	}
	listAppender.start();
	}
	public void after() {
	listAppender.stop();
	resetLoggingContext();
	}
	public void record(LogLevel level) {
	this.level = level;
	}
	public void recordLoggingForObject(Object sut) {
	Class type = sut.getClass();
	recordLoggingForType(type);
	}
	public <T> void recordLoggingForType(Class<T> type) {
	loggingSources.add(type);
	addAppenderToType(type);
	}
	public boolean containes(String loggingStatement) {
	List<ILoggingEvent> list = listAppender.list;
	for (ILoggingEvent event : list) {
	if (event.getFormattedMessage().contains(loggingStatement)) {
	return true;
	}
	}
	return false;
	}
	public int size() {
	return listAppender.list.size();
	}
	private <T> void addAppenderToType(Class<T> type) {
	Logger logger = (Logger) LoggerFactory.getLogger(type);
	logger.addAppender(listAppender);
	logger.setLevel(level.internalLevel);
	}
	private void resetLoggingContext() {
	lc.reset();
	}
	}

view raw LogStub.java hosted with ❤ by GitHub

Logging in JUnit Tests

With a JUnit 4.7 rules it is easy to add logging support in JUnit tests. The example test bellow shows how to add simple logging for each test method and the result of the test.

	import static org.junit.Assert.*;
	import static org.hamcrest.CoreMatchers.*;
	import java.util.Stack;
	import org.junit.Before;
	import org.junit.Rule;
	import org.junit.Test;
	import org.junit.rules.MethodRule;
	import org.junit.rules.TestWatchman;
	import org.junit.runners.model.FrameworkMethod;
	import org.slf4j.Logger;
	import org.slf4j.LoggerFactory;
	public class SampleStackTest {
	static final Logger logger =
	LoggerFactory.getLogger(SampleStackTest.class);
	@Rule public MethodRule watchman = new TestWatchman() {
	public void starting(FrameworkMethod method) {
	logger.info("Run Test {}...", method.getName());
	}
	public void succeeded(FrameworkMethod method) {
	logger.info("Test {} succeeded.", method.getName());
	}
	public void failed(Throwable e, FrameworkMethod method) {
	logger.error("Test {} failed with {}.", method.getName(), e);
	}
	};
	Stack<String> stack;
	@Before public void createEmptyStack() {
	stack = new Stack<String>();
	}
	// the sample test demonstrate logging of a succeeded test.
	@Test public void pushOneElementInEmptyStack() {
	stack.push("Item One");
	assertThat(stack.size(), is(1));
	}
	// the sample test demonstrate logging of a failed test
	// with an EmptyStackException
	@Test public void popElementFromEmptyStack() {
	stack.pop();
	assertThat(stack.size(), is(0));
	}
	// the sample test demonstrate logging of a failed test
	// with an AssertionError
	@Test public void sizeEmptyStack() {
	assertThat(stack.size(), is(1));
	}
	}

view raw SampleStackTest.java hosted with ❤ by GitHub

For more details about logging in JUnit tests see the blog post "JUnit 4 Test Logging Tips using SLF4J".