Software metrics are usually classified into product metrics, process metrics and resource metrics. Product metrics focus on finished artifacts like delivered software and documentation. Process metrics measure developer activities, development paradigms, tasks and milestones etc. Resource metrics measure inputs like people and hardware. Everything is measured: lines of code, binary size, number of classes/functions/interfaces, code coverage, commit history, documentation size, bug fixes, load time, performance, quality, reliability, cost, effort, usage etc. I'm mostly focused on direct metrics of available source code and executable itself: size and complexity.

Code size

This is the most straightforward and direct metric since it is based on counting lines of code. Counting lines of code (LOC) is de facto standard metric for the size of piece of software (or more accurately - program code). Variants on LOC are KLOC (kilo/thousands lines of code), MLOC (million lines of code) and SLOC (statement lines of code). Other measures can be based on LOC and provide more insight into analyzed source code, like number of errors per LOC etc. LOC-based metrics are popular because they are easy to compute and result in absolute measure of source code size. Alternative similar metric can be based on number of classes, methods or components. LOC is one of the most widely used metrics for developer contribution or productivity. Also, LOC per code commit or LOC per day are widely used metrics for developer activity.

Size metric like LOC is derived from static code analysis. Dynamic analysis during program run-time can be used for measuring "dynamic size" of running software. Dynamic size is closely tied to performance, e.g. number of objects, requests or transactions can provide valuable insights of software size or scale.

For those who are solely interested in LOC, popular web service Ohloh has large database of FLOSS projects along with statistics such as LOC, cost estimate and commit history. Also, check Wikipedia page about LOC.

Functional size

Functional size is calculated by quantifying the functional user requirements. Example of functional size metric is Function Point Analysis (FPA). FPA is used for estimation of project cost, duration and staffing size in early stages of development. FPA categorizes functions as data (internal files, external interfaces) or transactions on operating on accessing data (inputs, outputs or inquiries). For instance, external interface corresponds to data referenced (and not maintained by) application, while inputs are transactions where data enters the application. Since function point doesn't map to any physical attribute of a software, it is not easy to compute FP from program code or commit log. However, FPA is valuable because it measures software from user's perspective without being tied to specific technology.

For more info on FPA check this link.

Complexity

Complexity is usually defined as a degree of understandability and verifiability of some system. Although there are many other definitions of software complexity, I will focus on structural complexity (as opposed to psychological complexity) of static program code and dynamic execution graph. Software complexity is usually correlated with software size -- more LOC means more complexity (also, more complexity means more errors). McCabe metric and Halstead’s metric are historically important and commonly used code complexity measures. McCabe measures number of nodes and edges in directed graph derived from control-flow representation of the program. Halstead measures number of operators and operands and gives formulas for calculation of program volume, difficulty and effort (among others).

Other metrics include measuring structural complexity like number of directly invoked modules and number of inputs/outputs passed. These metrics include calculating fan-in (number of components that invoke current module) and fan-out (number of components directly invoked by current module). Coupling represents interconnections of code components. E.g. Information flow metric (INFO) calculates component usage as function of the number of calls into and out of an element. Opposite of coupling is cohesion or intradependence of code components (http://bit.ly/3rYNy). Besides already mentioned parameters, complexity can depend on software age, refactoring and maintenance process, coding style, tools used for development, experience of developers etc.

Dynamic complexity metrics are based on run-time execution of program code and they measure reference counts, dynamic trace analysis, loop executions etc. For instance, tools for profiling and tracing can find critical sections in program code and rank those sections by usage.

Good overview of code complexity is available here.

Tools

Here's a list of tools I compiled while I was gathering material for this post:

• CyVis - Software Complexity Visualiser
• GNU Complexity - Measure complexity of C source
• PyMetrics | Free Development software downloads at SourceForge.net
• Eclipse Metrics Plugin - State Of Flow
• Sonar
• ccm: analyzes c/c++, c# and javascript code
• C. M. Lott: Metrics tools for C/C++
• SLOCCount
• Collection: LOC Metrics - Alternative Tools
• Collection: Software Testing FAQ: Static Analysis Tools

Trends

I'm interested in general trends in software. Software today is based on layering principle where each component depends on layers (or stack) below. This pyramidal structure of one abstraction over another has tendency of software size reduction as abstraction grows. Program code is obviously getting smaller going up this pyramid; progression from Linux kernel and its 15 million lines of code to mobile applications and then to some small mashup script is clear evidence of this trend. Of course, components in layers themselves are becoming bigger; for instance, Linux kernel grows at rate of approximately 1 MLOC per year (http://bit.ly/X5LZB and http://www.ohloh.net/p/linux).

What about complexity? Complexity of whole stack is increasing. If for nothing, then for mere size of entire software pyramid. Structural (and psychological) complexity is decreasing going up the pyramid. You will probably need less time, experience, components (operators, data structures, functions...), interconnections and interfaces for web development than for kernel development.

With decreased size and less complexity more developers are able to join the game. With millions of developers arises question of developer contribution. More on that in next post.

Recently I'm reading a lot about visual programming languages with flow-based programming paradigms and since I'm obsessed with RSS feeds and web streams interoperability, I figured it would be fun to implement a subset of Yahoo!Pipes in Pure data (pd).

This project is just a proof of concept assembled using pyext for extending Pure data in Python. Additionally, I'm using existing RSS parser for Pure data. A simple example that fetches Twitter feed, filters on given text and truncates feed's items looks something like this in pd using pdpypipes:

Equivalent pipe in Yahoo!Pipes is available here.

This is not a full implementation of RSS feed mashup engine because I'm filtering only feed item's titles. Code can be improved by filtering structured feed data or passing around full XML feed. Code is available on Gitorious if you wish to implement additional operators or any other feature.

I just finished overview paper (in croatian language) on actor model and Erlang programming language:

Model sudionika i programski jezik Erlang

Writing tests as you write code is good strategy for writing clean code and building stable product. In this post I'll give a quick overview of my Django development workflow.

Doctests

Let's assume we're working on simple blog-like web application and we want to add search form. We will start with doctests and then incrementally refactor to unit tests:

• First we think about search interface. Search form is simple text field with optional submit button. Users submit search query to some URL where we capture and validate query, filter posts and return results. Simple.

   

• Next we write simple doctests for testing searching interface:

  Copy and paste code into tests.py file of core app. Run test with ./manage.py test core:

  Oops! Test failed. Django was not able to find URL named search. This is good because we know exactly what to do: URL pattern for searching.

• Add following line to urls.py:

  Run tests again:

  Test failed again. But, just like before, we know what is missing: search view in blog.core module.

  Let's run tests again:

  We have green testing bar! Now let's implement actual search.

• Remember: test first! Create couple of posts in testing database and write test:

  Of course, test are red now:

  It's time to revisit our searching code:

• Simple post searching can be implemented using filter method:

  And tests are green again!

In a nutshell: (1) write simple test for URL availability, (2) add URL patterns in urls.py and view function, (3) test trivial case and, (4) add more tests, (5) refactor implementation ...

Unit tests

Doc tests are fine for small projects. For larger projects you should switch to unit testing. Unit test are real code (doctests are code too, but in form of one large string hard to refactor and debug). Tests from this post can be refactored to Unit tests easily:

Conclusion

Good practice is writing tests while writing code (or better, before) and running tests before pushing changes to repository. My workflow forces me to think about interface first, write simple test and incrementally implement functionality. Developing using this methodology improves programming process and code quality while building test suite that is always available for validating code base consistency and improving programmer's self confidence. There's more to testing than just writing tests: fixtures, regression testing, mocking, browser/gui testing... Stay tuned..

What is erl-metafeed?

Erl-metafeed aims to become a scalable mashup engine for web feeds. The basic idea is to allow easy deployment of erl-metafeed mashup hubs in the cloud and to provide support for real-time web protocols. Similar services like Yahoo Pipes are GUI oriented and many users consider this approach to be overkill for simple pipes. Additionally, the erl-metafeed target audience are users who prefer textual interfaces over visual data-flow tools.

Similar to Yahoo Pipes, erl-metafeed enables creation of custom metafeeds. Metafeeds are analogous to pipes; they take one or more feeds as input and produce a single output feed by piping input feeds through series of filters. In erl-metafeed, metafeeds are created using custom query language to define input feeds and apply operations on those feeds. Once the metafeed is created, it can be retrieved from a unique URL as any other web feed.

Background

The emerging real-time web is based on feeds - streams of real-time information usually implemented using RSS/Atom protocols. Since real-time content is torrential, web users who follow a large number of web feeds usually overwhelm feed-readers (and themselves) with content they are not able to handle in a reasonable time.

Scalable Infrastructure for real-time feeds leverages web push. Real-time push operation is based on a distributed infrastructure of loosely coupled hubs that act both as subscribers and publishers of streaming web feeds (see PubSubHubBub).

Why Erlang?

Erlang is used for its concurrency support and soft real-time execution. Metafeed processes map nicely to Erlang processes and the query language interpreter used for metafeed definition was easy to code in a functional language.

How erl-metafeed works?

Figure 1. Erl-metafeed logical architecture

The erl-metafeed core handles web front-end and pool of metafeed processes. Users create metafeeds via a web interface. The core back-end analyzes user input and spawns a new metafeed process with the user's query. The created metafeed is available at a unique URL. Every time users hit the metafeed URL, a related metafeed process reads feed data from a database. The aggregator compoment syncronizes remote feeds with feeds in database. When the feed in the cloud updates, erl-metafeed re-evaluates all linked metafeed processes.

Query examples

Simple example of metafeed query specification that returns 3 most recent items from feed at some URL:

{tail, {3}, {fetch, "url"}}

A little bit more complex query that filters merged feed created from feeds at url1 and url2:

{filter, {contains, "some_text", ["title"]},
{union, {}, {
{fetch, "url1"}, {fetch, "url2"}}}}

Future work

Erl-metafeed is still a rough prototype. There are numerous bugs to fix and features waiting for implementation. This is not a project roadmap, but rather a list of things waiting for implementation:

• user authentication
• RSS/Atom remix
• PubSubHubBub client/server
• REST API
• easy deployment on Cloud
• handle JSON feeds on input

Source code and contribution

Erl-metafeed is released under AGPL licence and source code is on gitorious.

Currently I'm refactoring one of my Django based web apps to support different languages using Django i18n. However, I had some issues with translation of strings in Javascript files. Here are my observations on setup of i18n in JS code

Note: If you are new to Django i18n, please read here how to activate i18n support in Django.

• Because of security precautions built-in, add your project name to INSTALLED_APPS in settings.py. (see documentation)
• Create URLs patterns for i18n. Make sure that domain matches name of translation file, and packages contains project name listed previously under INSTALLED_APPS. Extend URL configuration file with something like this:
• Include link to Javascript catalog to your base HTML template:
• Create separate language files for regular (HTML and Python) files and JS files. You don't want to expose project-wide translation strings in JS for efficiency reasons. The translation strings could be specified in django.po for regular, and djangojs.po for Javascript files.
• Finally, you can use gettext function directly in JS code:

I hope you find these notes useful for your own i18n support in JS files!