EDITOR'S EYE

The World of Software Development.

June 2007

I get mail. Usually what might be described as "hate mail" or a note from my mother asking why I haven't emailed her. But occasionally I do get a real gem. Today's gem is from frequent Dr. Dobb's contributor Tom Thompson who checked in about the iPhone. Having written all kinds of applications for a long time, Tom knows what he is talking about when it comes to (1) mobile phones, and (2) anything Apple. That pretty much says "iPhone," wouldn't you think?

So here's Tom's musings about the iPhone:

It's going to be interesting seeing how the iPhone plays out. First and foremost, it's got to be a great phone. By great phone, I mean just that: it's got to receive and place calls reliably. If it can't do that due to the hardware or it crashes, then it's game over. If they've got that issue taken care of, then everything else is icing on the cake.

The iPhone has limited memory, no SIMM card, and no slots for memory or peripherals. Chris Sorenson, a Microsoft executive, said it was a "closed device." Darn tooting it is.... For the product launch and probably the first year Apple should have that thing closed. Nobody can download apps or obtain them from a memory card for a reason -- you don't want an errant app crashing the thing or turning on the transceiver. This isn't a PC we're talking about here, it's consumer electronics. I'm not saying that Apple shouldn't open the iPhone to third-party developers, but they're going to have to wait. And as for Steve Jobs saying you can write "Web 2.0 applications" for it, come on Steve, let's be honest -- those are just fancy scripts running in a browser.

The iPhone has already scored one big win: It's made phone OEMs rethink the UI. With the touchscreen, Apple got rid of the tyranny of the keypad/push-button interface. I've written simple example apps and a game for cell phones, and trying to deal with that minimalist UI was tough.

There's also tremedous interest in the iPhone because it promises to break the vagrancies of the carriers on data services. People are frustrated over carriers who disable certain features in phones to suit their own purposes, download only their apps, or curtail how you browse the web. If Apple's arrangement with AT&T makes them money, it stands to place some of the choice back into the hands of the consumer. The one fly in the ointment that I see here is that to load music, you have to "side-load" it from a PC via iTunes. The music restriction has to be in response to an objection from the RIAA, because AT&T would rather you download the music over their network and make money off you. Instead, consumers have to go to iTunes to do the job. Apple doesn't have a problem with that, as it further promotes iTunes as the front-end to a digital hub for phone, TV, iPod, and computer.

The iPhone doesn't have 3G technology, so it's going to be interesting to see how it holds up for extensive browsing and other data operations. Apple engineers probably went for the middle ground on the hardware to hit a price point and for reliability. Same on the limited memory. It's good enough to hold a few apps and songs, but not enough drive up the price. The GSM is a good idea, since it's available worldwide. The one big hitch in the hardware is the non-removable battery. Road warriors who make heavy use of their mobile gear sometimes swap a spent battery with a fully charged one, and worry about charging the exhausted battery later. You don't have that option with an iPhone. Again, it was probably done to keep the system closed, but I think they should have thought about usage patterns a bit more on this one.

If the iPhone is a success -- and that is a big if right now despite the high interest -- Apple will want to open it up. How they're going to do that will be interesting. Steve would like you to write Mac OS X apps for it. All well and good, but there are other factors at play here. First, the IT departments are going to initially balk at iPhones, and not because it's "irrelevant to business users" as Mr. Sorenson says. It's going to be because the IT guys will want to validate that the iPhone isn't a security risk. Steve doesn't like Java. I can understand part of that, because Java has such horrid UI elements, although it's been getting better at adopting the UI of the host operating system. UI aside, from my personal experience Mobile Java is pretty good at making small yet powerful, crash-resistant applications. Java has good security, and a number of enterprise applications will probably require Java clients. It's easy enough to put Java and Mobile Java on the device. It would also allow the iPhone to tap into existing, field-tested mobile apps and games. As much as Steve would like a whole raft of Mac OS X apps written for it, he's not going get the iPhone adopted in the enterprise unless Java is on the iPhone.

Nice letter Tom. Thanks.

Posted by Jon Erickson at 02:19 PM  Permalink |

You knew it would come to this -- blogging about blogging. But what is less known is that blogging has become a tool for public agencies to improve internal communications and speed the flow of information, at least according to a new report authored by David Wyld, Director of the Strategic e-Commerce/e-Government Initiative at Southeastern Louisiana University, and published by IBM.

Entitled The Blogging Revolution: Government in the Age of Web 2.0, the report details details blogging activities by members of Congress, governors, city mayors, and police and fire departments. In addition, the report assesses blogging in corporate America, with a survey of top executives who blog. For instance, General James Cartwright, commander of the U.S. Strategic Command (and recently nominated as the next Vice Chairman of the Joint Chiefs of Staff) has sponsored a real-time, secure blog to connect generals and warfighters. "The metric is what the person has to contribute, not the person's rank, age, or level of experience," says Cartwright. "If they have the answer, I want the answer. When I post a question on my blog, I expect the person with the answer to post back."

According to the report, blogging has moved from the purview of teens and college students to mainstream businesses and government. Every hour, more than 54,000 posts are made to blogs for a total of 1.3 million new blog posts each day. All in all, the blogosphere continues to double in size every six months.

"We hope this report both informs and inspires public managers across government to consider ways of engaging in the new world of Web 2.0 to improve citizen access to public services, as well as to enhance democracy in our society," says Todd Ramsey, General Manager, IBM Global Government Industry. "Blogging is no longer a fad. It is becoming a key tool in industry for communicating and collaborating both internally with employees as well as externally with customers. I anticipate that government agencies will continue to become rapid adopters as they see the power of Web 2.0 to help them address increasingly complex challenges that they face."

Actually, Wyld's examination of blogging is done in the context of Web 2.0 in general, although he focuses on interactivity. He sees blogging as a growing tool online citizen engagement with public officials. Wyld also points to other Web 2.0 technologies, such as wikis and 3-D immersive experiences like Second Life.

Posted by Jon Erickson at 09:30 AM  Permalink |

I'm beginning to think that IBM is serious about all this collaboration stuff. First it was IBM Rational with its Jazz.net, then it was IBM Lotus with its Quickr team collaboration tools. Now it's Many Eyes.

Created by the Visual Communication Lab at IBM Research, Many Eyes is a new social kind of data analysis that's based on the collective power of human visual intelligence to find patterns in large data sets.

Many Eyes lets users upload and view large amounts of complex data. Communities of people can then explore, discuss, and collaborate on visual content. Current data sets and visualizations reflect a diverse range of interests, including scientific (temperature and CO2 levels over time), political (U.S. political party affiliation over time), global (fertility rates around the world), and financial (venture capital investment in Europe).

For example, users are currently uploading to the Many Eyes site publicly available or personally gathered data regarding local or global issues -- tax rates, crime statistics, global warming, water safety, healthcare, and the like. The information is displayed in dynamic visualizations that allow for real-time knowledge sharing.

What's also interesting is that IBM has contributed Many Eyes for use during the Organization for Economic Co-Operation and Development World Forum (OECD) on Measuring and Fostering the Progress of Societies. The technology allows for the democratization of complex data by enabling users to upload any type of data and collaborate on the analysis of that information.

IBM and OECD representatives are of the mind that social computing software like Many Eyes will help people from all over the world make more informed decisions and create a new level of civic dialogue about the issues affecting them in their communities, their countries and at a global level.

"Nations, institutions and enterprises must work together to create a climate and culture for innovation; one that embraces cross-border, cross-organizational and cross-disciplinary collaboration," says IBM's Nick Donofrio. "Innovation is more than just the availability and accessibility of information technology; it is fostered when information technology combines with deep insight to create new value for business, government, academia and our global society."

Posted by Jon Erickson at 01:43 PM  Permalink |

Can you remember the first time you came across the Traveling Salesman Problem? "Given a number of cities and the costs of traveling from any city to any other city, what is the cheapest round-trip route that visits each city exactly once and then returns to the starting city?"

Yi-Chang Chiu, an assistant professor of Civil Engineering and Engineering Mechanics at the University of Arizona, probably can. While not necessarily tackling the Traveling Salesman Problem, Yi-Chang Chiu has cranked up a notch or two investigations into traveling from one place to another. In his case, Chiu is interested in moving people -- lots of people, not just a single salesman -- in response to some catastrophe, such as a terrorist attack or natural disaster.

"Solving large-scale evacuation problems is overwhelming," Chiu said. "No one can just sit down with a map and draw lines and figure out the best answer to problems like these." (Now he tells me. Pencil and paper was exactly the approach I took the first time I saw the TSP. And I quickly understood the benefit of having a computer handy.) So instead of pencil and paper, Chiu and his team have developed software that can react to a situation in real time, adjusting as conditions on the ground change.

The next generation of the software, currently under development by Chiu, Pitu Mirchandani, and Mark Hickman, is called "Multi-Resolution Assignment and Loading of Traffic Activities" (MALTA), and is designed to respond minute-by-minute to real-time emergencies via parallel processing.

The software package depends on detailed traffic census data that is collected by state and city transportation departments in conjunction with real-time traffic surveillance data. It then considers decisions each driver might make on factors such as when to leave, which route to take, if they listen to radio reports and change their route, if they are slowed by congestion and change routes, or if they react to freeway message boards that carry routing advisories.

Chiu has also used the software to model "value pricing" on toll roads.This concept is uses a sliding toll scale to manage congestion. As traffic increases, the toll increases incrementally to a maximum amount. This information is broadcast to drivers in various ways, with the hope that they will choose a different route, use public transit or delay their trip.

But value pricing on toll roads isn't an academic research project. For instance, based on the results of a seven-month pilot project in Stockholm, Sweden has adopted a road charging system designed by IBM and the Swedish Road Administration. This project saw peak-time road traffic congestion dramatically reduced, air pollutants cut by up to 12 percent, and public transport usage increased by around 40,000 commuters a day. IBM will proceed with the city-wide road charging roll-out in August 2007.

The Stockholm system -- the largest of its kind in Europe -- has 18 barrier-free control points around the charging zone equipped with cameras and a beacon system to identify vehicles and provide evidence to support the enforcement of non-payers. Payment channels include automatic direct debit, a Giro system at banks, over the Internet, and at retail stores such as 7-11.

For more information on road use charging, see the IBM paper entitled Driving the Future of Road User Charging.

And for hands-on (but not behind the steering wheel) experience traffic scheduling, try Mark Hickman's Transit Scheduling Game.

Posted by Jon Erickson at 10:22 AM  Permalink |

Regular readers of Dr. Dobb's Journal likely recall the April 2007 article Programming the Cell Processor. It was about, well, programming the multi-core Cell processor -- the processor that is famous for being the CPU(s) inside Sony's PlayStation3.

At this point it's worth noting that the authors of the article -- Daniele Paolo Scarpazza, Oreste Villa, and Fabrizio Petrini -- are affiliated with the Pacific Northwest National Lab (PNNL). But it is becoming clear to me that they do more than fool around with PlayStation3s and write articles for Dr. Dobb's. Not that I have any inside scoop, but my guess is that they have something to do with the new Computational Center of Excellence that PNNL and Mercury Computer Systems have cooked up. The Center will utilitze contributions from each partner, including hardware, software tools, and middleware, newly developed algorithms, and dedicated personnel.

"We’re excited to be working with PNNL, and about the possibilities of applying multicore computing technology to enable the development of economically viable computing solutions to previously intractable problems," said Jay Bertelli, of Mercury Computer Systems. "Early results from our collaboration show that, together, we can analyze streaming data in real time, which has been a critical challenge for data-intensive computing. Our goal is to open the door for new applications."

In the areas of defense and security, the computing power could be used on unmanned aerial vehicles (UAVs) to partially analyze incoming data onboard. Equipment on such platforms needs to be minimal in size, weight, and power. Multicore processors consume relatively low amounts of power while processing complex and large amounts of information. With the right software, they potentially provide the ideal fit for computationally intensive applications. Multicore processing could also improve the efficiency of cyber security for large computer networks. For example, rather than having a system that collects millions of pieces of information and then sends it to a central location for processing, the analysis could be done at a sensor that acquires or monitors the data. In the past, the processing speed needed to analyze the mountains of security data that today’s technology generates has not been available in a cost-effective suite of hardware. With more power in a compact form, a laptop-size supercomputer could become a reality for surveillance in multiple locations enabled by portable, real-time processing.

Mercury and PNNL intend to expand membership in the Center of Excellence to investigate computer technologies that include combinations of field-programmable gate arrays (FPGAs), multicores such as the Cell processor, GPUs, analog-to-digital converters, and software tools required for high-productivity development.

Posted by Jon Erickson at 04:53 PM  Permalink |

Collaboration has been on my mind recently. And on the minds of the folks I work with too. (They always get worried when I start thinking.)

For one thing, the upcoming August 2007 issue of Dr. Dobb's Journal is about collaboration, particularly in terms of globally distributed software development teams.

In the August issue, for instance, Jack Repenning, CTO of CollabNet (a company that knows something about collaboration) talked about "on-demand" collaboration and the close affinity between collaboration and open source, Khurram Nizami of IBM Rational focused on teamwork (clearly one of the most important topics in collaboration), and Steffan Surdek, also of IBM Rational, added a note on the challenges of distributed development.

And speaking of Rational, I also got an earful of collaboration at the Rational Software Development Conference last week, when the likes of Erich Gamma and Grady Booch, among others, shared their take on the topic. Grady, you recall, wrote a seminal article for Dr. Dobb's Journal entitled Collaborative Development Environments. And Erich demoed Jazz.net, an open commercial community project designed to improve team agility and collaboration.

But just when I thought I had put all this deep-thinking into collaboration behind me, Mindquarry, an open source collaborative software house I'd not run across before, released Mindquarry 1.1 -- a platform for file sharing, task management, team collaboration, and Wiki editing that's available for free download.

Version 1.1 features include:

• A "time machine" activity timeline that lets teams view older versions of files, Wiki pages, and tasks as a first step toward an ultimate undo/redo for all team content.
• An "interlink" that links to all files, tasks or other Wiki pages from within the Wiki. The link window also contains a search option to find items to interlink.
• The ability of team partners to insert tables and images into Wiki pages as easily as inserting these files in office documents.
• An Easy Tasks and Files Manager that lets users work online or offline on a Mac, Windows, or Linux. The feature synchronizes files and tasks as soon as users are back online.

"Mindquarry has reached a mature level of software quality allowing us to continuously extend our tool set for information sharing within teams," adds Mindquarry's Stephan Voigt. "Our goal is to empower dynamic teams to collaborate anytime, anywhere with anyone."

From what my co-workers tell me, that's about all the deep thinking I'm allowed for today. So in the spirit of teamwork and collaboration, maybe its time for a nap.

Posted by Jon Erickson at 01:47 PM  Permalink |

My fondness for puzzles is well known. What is less well known, but probably more expected, is my total inability to solve the bloody things. Crosswords. Sudoku. Jumble. I work on them throughout the day, keeping a pencil handy and filling in a square everytime I walk by. I'm no Mark Nelson, but I do the best I can.

Which is one reason why, from time to time, puzzles like Sudoku have found their way into Dr. Dobb's in the guise of articles like Gigi Sayfan's Maven: Building Complex Systems or even in this space with post such as I'm Puzzled.

And it is the reason I found Agnes M. Herzberg and M. Ram Murty's paper Sudoku Squares and Chromatic Polynomials so interesting. In their paper, Herzberg and Murty turn to graph theory in systematically analyzing Sudoku puzzles. In the process, they also find that analyzing Sudokus leads to some unsolved problems in graph theory.

Of course, this is also a topic that Eytan Suchard, Raviv Yatom, and Eitan Shapir covered in their Dr. Dobb's Journal article Sudoku and Graph Theory.

In the context of Sudoku, a "graph" is a collection of nodes connected by line segments. Sudoku has 81 squares (nodes) in a 9x9 Sudoku puzzle (graph). The twist that Herzberg and Murty take is to identify each of the numbers 1-9 by a different color. In a Sudoku graph, two nodes are connected by a line segment if the two squares they represent are in the same row, column, or 3x3 subgrid. Since no row, column, or 3x3 subgrid can contain more than one instance of each number, the graph will have no connected nodes of the same color. (For example, suppose you represent 1 with red. Two red nodes connected with a line segment would mean a row, column, or 3x3 subgrid had two 1s in it, which is a no-no in Sudoku.)

According to the American Mathematical Society, in graph theory, a colored graph with no connections between same-colored nodes is called a "proper coloring." What Sudoku solvers do every day is try to extend a partially-colored graph (the original puzzle with open squares means the graph representing it has yet-to-be-colored nodes) to a proper coloring.

With this analogy between Sudoku puzzles and graphs, Herzberg and Murty use graph theory to prove theorems about Sudokus. For example, they prove that the number of ways of extending a partial coloring of a graph is given by a polynomial. If the value of this polynomial is zero for a given Sudoku puzzle, then the puzzle has no solution; if the value is 1, then the puzzle has only one solution; and so forth. They also prove that, in order for any Sudoku puzzle to have only one solution, at least 8 of the 9 numbers must appear as given entries in the puzzle; if only 7 numbers are given, then the puzzle has at least two solutions. And this brings up an unsolved mathematical question: "It would be extremely interesting to determine under what conditions a partial coloring can be extended to a unique [proper] coloring," Herzberg and Murty write.

Some Sudokus are harder than others, with the really difficult ones containing very few given entries. What is the minimum number of given entries needed to ensure that a puzzle has a unique solution" Herzberg and Murty give an example of a Sudoku puzzle with just 17 given entries that has only one solution. So the minimum number is at least 17. But could it be 16 -- or something even smaller. No one knows. You might think that a puzzle with many given entries is likelier to have a unique solution, but this is not necessarily the case they say. In their paper, they give an example of a puzzle with 29 given entries that actually has two different solutions.

Posted by Jon Erickson at 11:42 AM  Permalink |

If you're not careful, it's easy to start thinking that the push for concurrency and parallelism, being driven (for the most part) by the adoption of multicore processors, only involves the ususal suspects: programming languages like C/C++ and FORTRAN, and vendors like Intel, Sun, and Google. But that's not the case, as evident by the waves that Python is starting to make when it comes to parallel computing.

For instance, in the upcoming August 2007 issue of Dr. Dobb's Journal, Robert Bjornson, Nicholas Carriero, and Stephen Weston will present their article "Python NetWorkSpaces" which examines a new way of writing parallel programs. NetWorkSpaces is an open source framework for that's easy to learn, accessible via almost all development environments (including Java, MATLAB, octave, Python, Perl, and Ruby),and easy to use clusters from within scripting languages like Python, Matlab, and R.

But NetWorkSpaces isn't the only game in town, when it comes to Python and parallelism. Interactive Supercomputing (ISC) has unveiled Star-P 2.5 for Life Sciences, an interactive parallel computing platform that lets you code algorithms and models using tools like Python or MATLAB, then run them instantly and interactively on parallel high-performance computers (HPCs). Star-P 2.5 for Life Sciences addresses a number of performance issues key to life-science research, including the ability to work with data sets that up to 4 terabytes across 512 processors. Star-P 2.5 for Life Sciences also features a 200-300 percent performance improvement for Fast Fourier Transform (FFT) functions on distributed data, which are commonly used in life-sciences signal and image-processing applications. It also offers a tool that lets you see real-time graphical analysis of how well programs execute, allowing optimization of the code structure for better performance.

It's worth noting that ISC also supports Python as a standalone package with its Star-P 2.5 for Python. Star-P 2.5 for Python includes a Python client interface that lets you take advantage of Python-specific numerical libraries and functions, including NumPy and SciPy, Python programming extensions that add support for large, multi-dimensional arrays and matrices, as well as high-level mathematical functions which operate on these arrays. Additionally, many modules from the Python open source community can be run as parallel tasks, to speed up tasks that can be executed independently.

Star-P for Python lets you use any of Python's hundreds of functions in a task parallel computation, such as Monte Carlo simulations or "unrolling" serial FOR loops. Additionally, for data parallel computing -- operations involving compute-intensive operations on large distributed data sets -- over 50 of the most popular "blockbuster" functions commonly used in technical computing are included.

"Star-P support for Python will enable NumPy users to run their programs on a parallel server or cluster with a handful of trivial syntax changes," said ISC's Ilya Mirman. "Our goal is to enable scientists, engineers and analysts to write number-crunching programs in a comfortable high-level language, and then immediately run their code on parallel systems with the least amount of complexity."

According to a recent study of 600 Python users sponsored by ISC and conducted by Fletcher Spaght, more than a third of respondents (35 percent) said that running their Python applications on high-performance computers would yield significant improvements to their research capabilities. This isn't surprising when you considers that more than a third (31 percent) of Python program run times take more than an hour to complete, with 20 percent taking days to complete. At the same time, Python users' data sets are only getting larger -- with 20 percent exceeding 10 GB.

So now you have an idea of why the scientific community in general, and Python users in particular, are interested in the benefits of concurrency, and the power of multicore.

Posted by Jon Erickson at 10:21 AM  Permalink |

It's been my understanding that "what-if scenarios" were the domain of spreadsheets. But that was before a team of researchers at the University of Southern California's Information Sciences Institute (ISI) introduced "risk analysis" and the "Semantic web" into the mix.

ISI's Michael Orosz is leading the team which is developing a tool called Risk Analysis Workbench (RAW), which brings together web resources, information sharing, game theory, and artificial intelligence to the task of gathering and distributing the specific data necessary to perform the tree analyses and other problems of "what-if" risk analysis in a speedy and -- above all -- in a uniform way. Where "what if" comes into play is in how the risk-related questions are framed.

Considering the "risk analysis" part of the equation, it should come as no surprise that the ISI is working with the U.S. Department of Homeland Security (DHS) and USC's Center for Risk and Economic Analysis of Terrorism Events (CREATE) to build systems that help policymakers make more consistent and speedier decisions when quantifying risk estimates. The Semantic web -- systematic structuring of database information to make it more accessible over the Internet -- is a major part of the system.

CREATE is an interdisciplinary national research center based at the USC and funded by the DHS. It focuses on risk and economic analysis of the U.S. and comprises a team of experts from across the country, including partnerships with New York University and the University of Wisconsin at Madison. The Center aims to become the world's leading academic program for modeling the risks and vulnerabilities of terrorism, assessing the direct and indirect consequences, gauging their economic impacts, and evaluating the effectiveness of countermeasures.

RAW is still in development. The current version is RAW 2.0, which according to Orosz will be distributed in July as an alpha version. As an example problem, the ISI team is now working to integrate into RAW risk analysis resources used in foot-and-mouth disease research being undertaken at the Center for Foreign Animal and Zoonotic Disease Defense (FAZD), and will be reaching out to the other six centers for subject matter.

Posted by Jon Erickson at 01:29 PM  Permalink |