Dr. Dobb's Blogs

Evolving Architectures: Part VII

Fri, 15 Apr 2011 05:34:00 -0400

Parallel design and simplification are the yin and yang of architecture evolution.

Azure Worker Role Is Not There Yet

Mon, 28 Feb 2011 12:39:01 -0500

Microsoft’s foray into the Cloud service providers (a.k.a. Azure) tries to play both in the Infrastructure-as-a-Service (IaaS) and Platform-as-a-Service (PaaS) playgrounds (Kate Craig provi...

Azure Page Blobs vs. Liskov Substitution Principle

Fri, 18 Feb 2011 21:49:08 -0500

“Liskov Substitution Principle” or LSP is one of the basic Alon Fliess and myself gave an open day @ Microsoft Israel on Azure cloud. Alon opened with an introduction to Azure Cloud and the b...

Mixins in .NET

Tue, 14 Dec 2010 23:05:58 -0500

In object-oriented programming languages, a mixin is a class that provides a certain functionality to be inherited by a subclass, while not meant for instantiation...

Cloudoscope - a cost profiler for the cloud

Mon, 29 Nov 2010 01:04:20 -0500

I am very happy to announce that I am joining CodeValue. CodeValue, as our site says, is the home for software experts. Indeed, the company is built of a group ...

Random Hacks of Kindness

Mon, 15 Nov 2010 03:37:20 -0500

I don't usually go promoting events and such but I thought I'd make an exception for this one I've got the notice below from Vivi-Anne Kelly the Operational Lead for RHoK and I am bringing it as is...

SOA Patterns: Short Status Update

Wed, 06 Oct 2010 15:17:09 -0400

First off, In the previous post I published the Transactional Integration anti-pattern – if you need it for off-line reading you can also get it in PDF form.

I am currently writing the “3-tier SOA” anti-pattern and it seems that together with “Whitebox services” anti-pattern it will complete the anti-patterns chapter. The two other anti-patterns in the chapter are Nanoservices and the Knot

I’ve also started writing the composite-front end pattern (e.g. portals, prism etc.) but mid-way I thought about it and stopped. I basically realized that there are a few patterns that are pretty common to SOA on one hand, but you are much more likely to use a 3rd party solution that includes them than to realize them yourself. These include the composite front-end mentioned above, repository, orchestration, servicebus, service host that appears in chapter 2 (though I think that is marginal case) and maybe a couple of others. I am thinking in moving these to an “SOA infrastructure patterns” chapter. The open question I still have is would they need the same structure and depth as the other patterns. For example it might beneficial to emphasis usage, or expand the technology mapping etc.

As always, any comments are welcomed

By the way, if you are struggling with the architecture of your distributed system/SOA, I’d be more than happy to help you sort things out

SOA: Contracts, Events, and Ownership

Mon, 20 Sep 2010 02:25:27 -0400

I recently listened to Udi Dahan's excellent "Avoiding a Failed SOA" presentation from QCON London (it is about an hour long, but it is worth your time).

So Do We 'Build' Software?

Tue, 07 Sep 2010 14:15:59 -0400

Phew, August is finally over -- no, not because we closed xsights, not even because my wife and I had to entertain 3 kids on vacation. It was the renovation of our balcony and living room that made it a living hell. Oh well, at least it is over, and it got me thinking (at least when they weren't banging,breaking or otherwise making noise in general). There's this analogy that you "build software" and that is similar to "building houses" -- this is especially popular with software architect trying to understand what it is exactly then need to do :). The analogies usually have something to them but they only take you that far. For example (which I have to admit I also used in the past) for an analogy: If you are going to build a dog house then you need certain tools and skills (and most of use can pull it off); if you are going to build a house, you might be able to design some of it, but you'd need help with most of the design, and way more tools, flows, practices and help with the construction. If you want to build a skyscrapper, well that's something you'd definitely going to leave to professionals and it requirers yet another set of tools, processes and professionals. Software doesn't work that way. Sure a hobbyist may use other tools than the professional (even that's not always true) but with software you might set out to build a skyscrapper but start with a dog-house, then send into production a "dog-scrapper" and maybe one day end with a sports-arena because that is what the customer actually needed. You can't do that with buildings. An analogy I still like is that of building a new intersection in regard to introducing an architectural change (e.g. moving an enterprise to SOA) -- with the idea that the business would not stop and wait until you'd finish your change and you have to build detours, only close some lanes etc -- just keep the business and information flowing. Like the other analogy though it is still limited in it's applicability. This month, however, proved there are some similarities I didn't think about:

It turns out a building project can take twice as longer than planned.Why? Because the contractor took too much work and had his worked task switching a lot (half a day here, half a day there)
It turns out that even though there's a detailed plan new requirements can still come up as the project progresses -- both from customer requirements (adding stuff, exchanging stuff) or because of the realities of the project (height and angle problems that were not foreseen)
It turns out that people can cut corners only to find out it would only get them that far (and then have to re-do everything properly)
It turns out teams matters. We have two neighbors who are completely renovating. One is 2 months overdue vs. the other team that took a week to do what the other team needed a month to complete.

Oh well, while I still think software is closer to writing a story then it is for building a house, maybe there's merit in the building analogy after all... what do you think?

Evolving Architectures: Part V – Leaps

Sun, 01 Aug 2010 15:49:00 -0400

Looking So You Don't Have To "Leap"

Looking for New Opportunities

Wed, 28 Jul 2010 11:41:40 -0400

After 3 years working in xsights as VP R&D, it is now time to move on. I am looking for an interesting development management position preferably as VP R&D. Other interesting opportunities (e.g. a chief architect position) will also be welcomed. The position should be in Israel (Hasharon, Tel-aviv or Haifa areas) On a related note, I am also available for focused consulting engagements. By focused I mean deliverables oriented (not by the hour) engagements such as architecture evaluation, technical due diligence or architecture workshop. You can find more details here. Currently I am available for consulting world-wide. If you are interested you can grab a copy of my CV from here (English) or here (Hebrew) Last but not least, since we are closing my team is also on the look for new jobs so if you need some great .NET people please ping me as well You can contact me here

Evolving Architectures - Part IV - Design mechanics

Mon, 19 Jul 2010 14:08:15 -0400

If we want to evolve architectures or change an existing architecture in general, for that matter, it is important to understand design mechanics. I recently attended a seminar with Kent Beck that talked about "responsive design", where he provided a good definition for 5 strategies or types of design mechanics*:

Leap
Parallel
Simplification
Stepping Stone
"Just Do it"

Kent talks about these from a design and coding perspectives. Architecture is design but at a higher level and with more consequences system-wide so (I think) are are few nuances from how Kent sees this.

"Just Do it" is what you do when you don't exactly know where you are going and you so you're adding features and everything but you probably heading into a big ball of mud. If possible it is better to avoid working this way, but sometimes realities take us there anyway.
Stepping Stone is what you do when you aren't completely sure what needs to be done but you can imagine what would make the end-goal easier to reach. The way I see it, for architects that would mean an architecture proof of concept (spike), prototype of skeleton - the [intlink id="saf-architecture-evaluation-evaluation-code" type="post"]three ways to evaluate architecture in code[/intlink]
Simplification - is when you don't know what you want to do and it is too expansive to directly get to an end result. This is the basis of the iterative system. start small and gradually add features as you go. From an architecture perspective it is challenging to to get to a core simplified architecture you can grow later. This is useful as long as the changes in the architecture are within the expected growth path. when it is time to make a change in the architecture you'd need to go with the other strategies.
Parallel - is a technique to use when you know where you are going, but can't afford to stop/halt what you already have so you keep the two designs going at the same time and migrate to the new way gradually. This is probably the best technique to use to evolve architecture as it fully considers what you already have. An analogy I like to use in describing this situation is building a new interchange on a highway. The new way would be great, but you can't just stop traffic until everything will be in place so you've got to allow the old way (maybe not in full) until you can make the change safely - the same is true for a running business, nobody will halt the business until you'd get your new shiny thing up and running
Leap - is for when you know where you're heading and you can afford to stop everything until the new way is in place. From an architecture perspective it is a bit like starting from scratch (I'll expand on this in the next post). From a design perspective that would mean breaking the build until a few related changes are in place

The next post will expand a little on Leap, Parallel and Simplification

* Kent also discussed several other interesting subjects related to design like cohesion and coupling or this nice definition for design which are worth reading. ** Illustration by dipster1

Tidbits

Tue, 06 Jul 2010 11:55:12 -0400

New month, new round of tidbits. IASA Israel Chapter 2nd Meeting This Thursday (July 8th) at 17:00 Israel time, IASA Israeli chapter is holding its 2nd meeting. I am going to be there moderating the discussion on Architecture and Lean/Agile projects if you are around Raanana feel free to join SEI Webinar on Software Architecture Another software architecture related event going on on July 8th is a SEI webinar on What's software architecture by Rob Wojcik the webinar will take place on July 8 from 1:00 to 2:00 EDT. Looks like it is going to be an interesting listening if you are new to Software architecture. Business Analysis Agile survay Since my blog is also published in Dr. Dobb's I get a lot of PR requests from all sorts of places, most of them end in my Junk email (if the SPAM filters doesn't catch them first). From time to time something a little more interesting crops up. This time it was this survey ran by requirements.net , the International Institute of Business Analysis (IIBA) and capgemini. What's interesting about this survey is that 52% of the respondents were in business analysts (or BA management) roles and that 62% of the companies they work with had 1000+ employees. So we get some views of non-devs from large companies. What I see in these results that Agile (with a capital A) is very much in vogue so everybody wants to say they are practicing agile even if they just moved to an iterative cycle within the same old waterfall process or as the survey word it:

Comments from participants demonstrate that many in IT Management desire to maintain current processes and organization structure, while delivering iterative development cycles. Of course, this often results in a hybrid Agile-Waterfall methodology which is commonly referred to as "Scrummerfall" [ARGO: I guess that's pronounced 'Scrum-fail' :) ] or "Wagile."

Also it seems a lot of people don't really understand agile by thinking that SCRUM is a synonym for agile; thinking that product owner and scrum master are a logical fit to be the same person (46%) rather that dev and QA (only 11%); leaving the business analysts as "requirements authors" (42%) rather than having them as part of the "client team" (as Subject matter experts (21%)) or product owners (16%) etc. I guess the direction is what's important :)

Illustration by esc.ape(d)

Tier is a Natural Boundary

Tue, 22 Jun 2010 01:17:16 -0400

When talking about multi-tiered architectures, we need to remember that the tier boundary is significant. The tier boundary is where distribution happens and if you remember the "fallacies of distributed computing", you know not to take that lightly. A tier is a physical boundary (versus, for example, an Edge in a SOA which is a logical boundary within the service). The implications of that are numerous. For instance, you need to consider:

Trust--who do you let in?
Security--what do you send out?
Performance--you need to serialize to pass the boundary, and remote data is expensive to fetch.
Availability--what happens if you crash?
Manageability--can anyone see what's your state? Help you recover?
Temporal coupling--can you afford to make synchronous calls?
and many similar questions.

Yet many times people think passing a tier is as simple as passing a logical layer. I should know. I made this stupid mistake more than 15 years ago in one of the first distributed systems I designed. I planned this beautiful separation of the UI controls from the business logic (I didn't know it was called "MVC" and that someone else had figured it out ages ago, so I was pretty proud of myself). When you clicked on a button you just used metadata to say that BL should catch it. I had all this wonderful "infrastructure"that handled passing the call to its destination. But then we wanted to take this n-layer application and put the BL in an "application server" which will handle multiple clients. Oh--now we need to move events over the wire , handle calls from multiple unrelated clients, pass a lot of data back and forth, and what about security... you can imagine the fiasco. Thus, as Niels Bohr once said, "An expert is a person who has made all the mistakes which can be made in a very narrow field." But you don't have to make the same mistakes. Just remember that a tier is a natural boundary. You know what? You should think of a tier as the edge of a cliff, right there - at the end of your application. You know you want to be careful and not fall down.

[Originally posted Aug. 2006]

Who Needs an Architect Anyway?

Sat, 05 Jun 2010 23:05:08 -0400

Not all projects need architects. There, I've said it. Not all projects need architects and I am not talking here just about trivial projects.

Software architecture should start with Why

Mon, 24 May 2010 01:33:50 -0400

I recently saw Simon Sinek's TedX talk on Start with Why (see below) talking about leadership. But WHY am I telling you this? For one, it's a good talk on leadership and inspiration in itself (well worth the 18 minutes or so it would take you to watch it). The main reason, however, as the title says, is that it also pertains to software architecture decisions. Simon talks about "the golden circle" going from Why -> How -> What. Saying most people work from the outside in (rarely getting to the why) and leaders work from the inside our (starting with Why). It is true for us as well - Software architecture decisions should start with why We're not just building an SOA (What) using mule or nservicebus (How) we're doing that to build a flexible solution so that we can respond to changing business needs (Why). We're not just using a NoSQL solution (What) using Cassandra cluster (How) we're doing that to solve a database reliability and availability problem of managing a large dataset with high write (Why) to allow the business to support X millions of images (in xsights case) with Y concurrent users and search times less than Z seconds (a better Why) This is the essence of why it is a very good idea to describe [intlink id="quality-attributes-introduction" type="post"]Quality attributes[/intlink] in terms of functional scenarios. We want to make sure that an architectural solution will be tied and will answer a real business scenario within the system. We don't want to provide fault-tolerance or 99.999% uptime . That's not interesting. We need to define why we need high-availability, what will be the consequences of the downtime loss of a business transaction? loss of 100K dollars? loss of life? - for the last case maybe 99.999% uptime is not enough for the first case maybe a 95% uptime will do - start with why. It can save you money, it will help you get to the right requirements and it will help you justify your decisions

Nanoservices Anti-Pattern (PDF Version)

Tue, 18 May 2010 01:11:10 -0400

The formatting on the HTML version of the nano-services is a bit off (Word to HTML is so much fun) so I am also making it available as PDF.

If you don't remember:

Nonoservice is an Anti-pattern where a service is too fine grained. Nanoservice is a service whose overhead (communications, maintenance etc.) out-weights its utility."

Evolving Architectures: Part III Starting Out

Sun, 16 May 2010 15:39:00 -0400

Architectures' "Do's" & "Dont's"

SOA Anti-Pattern : Nanoservices

Wed, 28 Apr 2010 02:43:02 -0400

SOA Patterns book - your opinion needed

Wed, 21 Apr 2010 13:07:31 -0400

It has been quite awhile since I added anything new to the book. I have my reasons (some would probably say excuses :) ) mainly that finding the energy and time to write is very hard with a wife, 3 kids and a startup.

Anyway, I've been talking with Manning lately trying to figure out what to do with this project. I was quite amazed to learn that 1000 or so of you purchased the MEAP edition even though it only contains 5 chapters and haven't been updated in a long time. (by the way I've also recently learned that book pirating sites are offering the book for download, but that's another story). Anyway, we're trying to decide what we want to do and this is where I'd love to hear some feedback from you

1. Do you think the book is still relevant today?

2. Do you think the book should be restarted form scratch to reflect recent

3. Do we need to cancel the book and end this fiasco ? or push it though to completion?

4. How valuable do you find the information in the book so far?

I am ready to put in time to to add enough patterns/anti-patterns to make the book releasable. but since I know it takes oodles of time I don't really have I really want to know I am not wasting my time

Please leave a comment/drop me an email if you have anything to say

Thanks (and thanks for continues patience so far)

Yes to NoSQL

Thu, 01 Apr 2010 01:05:39 -0400

There seems to be some backlash building up against NoSQL with posts like Ted Dziuba ";I Can't Wait for NoSQL to Die"; or Dennis Forbes' ";The Impact of SSDs on Database Performance and the Performance Paradox of Data Explodification (aka Fighting the NoSQL mindset)";.

These are interesting articles to read and yes RDBMSs are not going the way of the dodo yet (I even said that in ";The RDBMS is dead";, which by the way, was written before NoSQL was coined, but I digress ). Nevertheless, other options, namely various NoSQL choices, prove to be easier to work with, and are a better fit in some circumstances.

Few time (ok very few times), that's because of sheer size (The ";Google";s) but what about the rest of us, should we just stick to good-old-RDBMSs?

If you are have a good understanding of RDBMSs (and/or a great applicative DBA working in your team) then in many cases the answer would be yes. I had a chance to review a couple of systems recently that deal with large amounts of data (in the Petabytes) both systems had a very good DBA teams working on the solution to a point that even ORM was not that important for the solution.

However, in many other cases NoSQL can be a better option. For instance let's take the Digg case Dennis Forbes talks about. Assuming, for the sake of argument that Dennis completely nailed Digg's requirements(*) and his single computer +SSD solution is a reasonable solution for Digg. Still the fact is that the Digg's team where not able to solve their problem with an RDBMS and the same team was able to pull it off with a NoSQL solution. So even if we say the team is mediocre (something they are probably not), NoSQL made it easier to solve a problem they couldn't have otherwise - that's a good thing in my book.

There are additional reasons to use NoSQL besides sheer size and better alignment with developers' mind-share though. The more important ones of those are to get cheaper (vs. comparable commercial RDBMS solutions) scalability and availability. Again if we look at the Digg switch to Cassandra they cited ";the lack of redundancy on the write masters is painful"; and the administrative overhead as main reasons.

In any event it It is important to keep in mind that NoSQL is not a silver bullet. You need to assess whether or not it is suitable for the problem you have at hand for instance NoSQL solutions place emphasis on different parts of the CAP theorem than RDBMSs do is this what you need? etc.

* I think that Dennis over-simplify Diggs situation since they also, for example, have heavy concurrent writes etc.

Architectural requirements Q&A

Tue, 23 Mar 2010 14:35:42 -0400

A few days ago I was contacted by Lianping Chen a doctoral researcher from the Irish Software Engineering Research Centre. Lianping is doing research on ";how to elicit architectural significant requirements"; and he asked me a few questions, which I though, might be interesting to a wider audience.

Evolving Architectures - Part II but Design is emergent

Thu, 11 Mar 2010 12:04:35 -0500

This is part II of a series on agile architecture.

Evolving Architectures - Part I What's Software Architecture

Mon, 11 Jan 2010 20:00:00 -0500