every creaking door sounds like whale song

Escape from Metropolis

2006-07-28T10:23:00.000+01:00

I've expanded on an old blog entry in this article: Escape Metropolis: Bridging the Divide Between Developers and Architects.

Assertions on architecture and strategy

2006-06-09T08:37:00.000+01:00

#1: A good balance is achieved between architecture and strategy when strategy is biased to the prophetic and architecture to the socractic.

#2: A prophetic architect is dangerous without constraint.

#3: An architectural organisation that is failing often undergoes a Constantinian shift. When it does, membership of the architecture community requires acceptance of certain dogma and new thinking and socratic behaviours are supressed. This is often associated with a rise in the unconstrained prophetic.

What is a software architect?

2006-05-29T22:23:00.000+01:00

It seems to me that architecture is a somewhat arbitrary term in IT, there is just design, construction and validation[1]. Someon can design a class, a module, a package, a set of packages, an application or a set of applications. As they move up the stack the amount of abstraction they deal with increases (since unless they're very smart they cannot hope to understand it all). At some point up that stack people seem to draw the line and label people doing design at that level as 'architects'. I thinks that is pretty much all there is to it.

The question of 'what is a software architect' then becomes 'where do we draw the line in that stack'. Of course, there are many stacks, all interelating. For example, there's a data stack (tables, application schema, logical models, conceptual models) and an infrastructure stack (switch configs, physical network diagrams, ....). Because different people operate in different stacks and at different points, they tend to define architects in terms of their location in a particular stack, and there is normally much confusion as a result.

One thing that has occurred to me is that the point at which people draw the 'architecture' line oftens seems related to the loss of 'singularity of purpose'. Developers and module designers are normally focused on one goal, with normally one stakeholder to manage. At some point they become team leaders or design leads (often called application architects). At this point life becomes that much more complex, singularity of purpose is lost as the job becomes more about weaving a path between all stakeholders and their views.

----
[1] order, breadth, depth and persistence dependant on the chosen SDLC.

Are you living in metropolis

2005-09-16T12:32:00.000+01:00

The IMDB plot summary for Metropolis (1927) is as follows:

[snip] humans are divided into two groups: the thinkers, who make plans (but don't know how anything works), and the workers, who achieve goals (but don't have the vision). Completely separate, neither group is complete, but together they make a whole. One man from the "thinkers" dares visit the underground where the workers toil, and is astonished by what he sees...

This seems to sum up the stereotypical divide between enterprise architects and delivery teams - as seen from both sides. That is, architects thinking developers lack vision and developers thinking architects have no idea how anything works.

Vendor Selection Patterns

2005-08-17T06:26:00.000+01:00

I've been involved in a number of vendor selection activities over the past few years. Recently I produced a paper called Vendor Selection Patterns as input into EuroPLOP. Unfortunately I failed to make it to the conference, but you may get some value from some of the lessons contained in this paper. I'll be submitting this to EuroPLOP next year and I imagine the post-workshop revision will be significantly improved. One of the really tough things was finding good references, the set of good technical books on the RFP process seems a bit thin on the ground.

Standards cause design fragility when used unwisely?

2005-06-20T07:24:00.000+01:00

Enterprise architecture is in someways like urban planning. You have systems (buildings) connected by infrastructure (road, rail, sewage etc) and developers and users have to live in that city being built. The city changes over time as new buildings are put up, old ones taken down or extended and the infrastructure evolves. Of course, a city isn't as plastic as software and software doesn't have to obey many physical laws. Anyway, despite these problems, let's assume the metaphor holds and see if we can learn something.

In Europe Le Corbusier had significant traction as an architect and wrote up his ideas on urban planning, which were fundamentally modernist in origin. His ideas centred around zoned standardised accommodation and infrastructure, one of his proposals for a section of Paris is below:

His ideas on urban planning resulted in utopian drawings of homogenous living units in homogenous cities, with urban zones and a grid based model. These ideas have been largely discredited and the results of applying his ideas in the UK, the tower block housing estate (e.g. at Roehampton), have failed with pretty disastrous social results. I believe one of the problems with this modernist approach, where a large area of a city is planned according to a single design approach/theory, is that they allow design failures to effect the entire planned area due to the homogenous nature of the approach. Of course, we could blame the modernist movement as a whole, but that seems harsh. In Chicago, Mies Van Der Rohe, whilst being a modernist, didn't get to build a Brasilia (which was based on Le Corbusier's ideas). He had his modernist intentions bounded. Go to the plaza area in Chicago he designed, and whether you like it or not, it is only one design amongst the many surrounding it. A failure in Van Der Rohe's design approach doesn't break an entire area as the urban plan has been far more eclectic in its choice of approach.

I am tempted, given my initial parallel of urban planning with enterprise architecture to draw a parallel between Le Corbusier's ideas and those of the breed of architect that want all of the enterprise architecture standardised on single choices. If we do this, and our chosen technology fails (e.g. a major vulnerability is found in the single chosen operating system or J2EE platform), then we have effected vast tracts of systems. Of course, standards provide many many benefits, but maybe the trick is to ensure there is enough heterogoneity in our city plans to protect against major design failures in our enterprise architectures.

Perhaps we have more to learn from todays urban planners than we may first realise.

Archaeology as a metaphor for understanding systems.

2005-05-30T12:02:00.000+01:00

A number of people have been using the archaeological metaphor as a frame within which to discuss the process by which we understand software systems. I've been discussing this with a colleague of mine on and off for a few weeks and we've come to the conclusion that while this metaphor helps, it isn't the whole picture.

Let's start off by defining software archaeology, and we'll use a dictionary definition to do this[1].

soft·ware ar·chae·ol·o·gy Pronunciation Key (sôftwâr ärk-l-j)
The systematic study of past software systems by the recovery and examination of remaining material evidence, such as code, tests and design documentation.

Using archaeology as a metaphor allows us to reason about how we recover and examine material evidence relating to software projects. As useful as this is, it is by no means the entire story. Many times there exist individuals to interview, and primary and secondary evidence to investigate. These types of activities owe more to historical practice then they do to archaeology. One of the reasons for this is that existing communities in organisations ensure that the investigation of systems is as much about people, existing practice and precedent as it is about tools and artefacts 'dug out of the ground'. As an illustration, consider the UK constitution. The UK constitution defines the form, structure, activities, character, and fundamental principles by which UK society, law making institutions and government institutions operate. Famously though, it is not written down. Instead it consists of a number of bodies, namely the monarchy, the executive, parliament and the civil service, supported by both precedent and law. A student of the UK constitution is rather like someone trying to understand a software system. The student has to talk to experts, study paperwork, identify 'urban myths', analyse past behaviours and cope with the fact that, whilst they investigate it, the constitution is changing. The I find this process, the process of historical study, to be a more compelling analogy than archaeology when considering practical techniques for studying software systems.

[1] Even the archaeological community hotly debate the definition of archaeology so this is a working definition, but it is not the only valid definition.

Modeling multiple business perspectives

2005-05-11T16:12:00.000+01:00

I've been involved in a conceptual modeling exercise that, whilst moving forward, would occasionally suffer from 'modelers' block. I've been using the following standard approach to help with this.

Problem

You need to build a conceptual model of the key business entities but are finding it hard to bottom out the meaning of these entities due to conflicting viers on the entities semantics.

By modeling the business you are modeling multiple concents, e.g. scheduling, settlement and trading. Each aspect of the business views the entities from different perspectives. You need to obtain agreement on the semantics of the entities and their relationships but this is tough with everyone viewing the model from different positions.

Solution

Model the business from each useful perspective and then reconcile the models.

During the modeling workshop explicitly communicate which perspective people are to view the model from. Work through the model from that specific perspective (an no others). Narrate the place of the key entities in the business process as you go. e.g. Trader under name of a legal Entity does a deal with a counterparty at a given time for an agreed price. This keeps things practical. Once you have got as far as you can from one perspective then switch perspectives. Start with a clean sheet, as building from one model (adding perspectives as you go) forces you to reconcile models 'on the fly' - which is very hard and prone to failure. Once you have models for each perspective ready, merge them together. Where they don't match, reconcile the perspectives through a workshop.

Resulting Context

By making the perspective you are modeling from explicit you have partitioned the modeling activity into a few easier modeling problems. When you recombined the different perspectives you forced the business to reconcile the different perspectives in a controlled way. The result being a conceptual model for the business that all aspects of the organisation can buy into. In addition, the different pre-reconciliation models tell a story of how the design was developed - and are useful when explaining the model to the unitiated.

Incorrect Application

Using too many perspectives.

Switching perspectives continually in a modeling session.

Known Uses

Syntropy uses different model perspectives (rather than business perspectives) to ensure modelers know what aspect of the modeling activtiy they are address.

Large global organisation is currently building a conceptual model of the business and has been using this approach with a good success rate.

Jackson describes using problem frames to control and frame thinking processes.

Arts & Crafts movement alive and well in software?

2005-05-10T07:10:00.000+01:00

I went to the International Arts & Crafts exhibition at the weekend. The items on display and outline of the Arts & Crafts movements sparked of a number of thoughts, two of which I'm going to expand on here.

A perspective

The Arts & Crafts movement (I am talking specifically about the UK movement here) feels to me very much like a 'middle class' activity. I hypothesise that the rise of an affluent urban class was a pre-requisite for the Arts & Crafts movement to have succeeded as it did. As industrialisation progressed a well off urban class started to form, a class that romanticised rural life[1]. This class's intellectuals, from an urban perspective, started to question the impact of industrialisation. Out of this Arts & Crafts was born. Urban craftsmen moved out into rural areas and learnt from country people. They applied those skills, mixed with the Arts & Crafts philosophy, to produce elegant works. The output was then imported back into the cities by the rich urbanites. These 'country designs' continued the romanticisation of the countryside by providing some of the raw material that allowed affluent city dwellers to construct a hyperreal version rural life. More cynically, they could sit in their solid practical Arts & Crafts arm chair and comfort themselves as industrialisation increased their isolation from the environment.

I could be well off the mark here since I have very little knowledge of that part of history.

Alive and well in software?

The V&A descibe the movement thus: "It was a movement born of ideals. It grew out of a concern for the effects of industrialisation: on design, on traditional skills and on the lives of ordinary people. In response, it established a new set of principles for living and working. It advocated the reform of art at every level and across a broad social spectrum, and it turned the home into a work of art."

and

"In Britain, the Arts and Crafts Movement flourished from about 1880. At its heart lay a concern for the role of the craftsman. Inspired by the ideas of John Ruskin and William Morris, it advocated a revival of traditional handicrafts, a return to a simpler way of life and an improvement in the design of ordinary domestic objects."

Doesn't this feel a little like some parts of software at the moment? There are workshops on beauty in software, rallying cries against Taylorism and Fordism. Everyone needs to get back in touch with the craft of software, just write code, do the simplest thing, test first. Are we at the start of softwares Arts & Crafts movement? Will we stand at the shore commanding the tide of industrialisation retreat? Will some of us spend more time crafting our products and taking more pleasure in the act of creation?

Did the Arts & Crafts movement make a lasting impact on people as a whole? We can certainly buy expensive William Morris wallpaper and import Mouseman blanket boxes from the North, maybe that is it. All we are left with are these vestigial organs. Maybe we are on a hide into nothing?

This is a bit tongue in cheek (I'm a big proponent of agile methods), but it is a mildly interesting parallel. Perhaps.

-----
[1] As industrialisation proceeded the English started to romanticise rural life. This has continued to this day.

Models, process and Zachman

2005-04-24T17:55:00.000+01:00

John Daniels ran an interesting workshop on models and perspectives. I'm pretty familiar with the source material for this, namely Syntropy. However, John went on to talk about how you move from model to model and what the motivation for a model maybe.

Intent/Perspective	Conceptual	S/W Specificiation	S/W Implementation
Sketch	1	2	3
Blueprint	4	5	6
Program	7	8	9

We ended up with a table like the one above, where intention is orthogonal to the model perspective. John Daniel proposes that you can characterise a method by the cells it populates either fully or partially. e.g. XP is a 3,9 method or, as someone joked, a 9,9,9,9,9.... method ;-) I thinks this is not only an interesting way of thinking about methods but also a good way to look at enterprise frameworks such as Zachman. If we consider Zachman we can map the rows to perspectives. Columns however are tougher since they indicate content (Who, What, Where, When, How, Why). If we leave intent as it is in the above table we can map content as a 3rd dimension. This gives a cube that can be used to characterise architectural processes. In the cube, each cell can contain partial or complete coverage for 0 or more of the content prescribed in the Zachman columns.

SPA 2005 Note - MDA

2005-04-24T17:53:00.000+01:00

Whilst at SPA 2005 I met someone from a previous company I worked at. Apparently they have been using MDA to great effect. They have a UML model of their aspect of the business. This model is annotated with OCL (defined by the analysts). The OCL documents the models constraints[1]. They generate XSD and Java off this model, the XSD defining the structure and the Java expressing the constraints. I'm interested in MDA and its applicability to the company I work at for some time, it was good to note that other practically orientated organisations were starting to use it.

They are using the following tooling (with XML as the export language from the UML modeling tool to the stuff below).

http://lci.cs.ubbcluj.ro/ocle/
http://xmlbeans.apache.org/
----
[1] Convergence - feels like OWL could work as well.

SPA 2005 Note - Outsourcing

2005-04-24T17:51:00.000+01:00

The workshop went through the standard set of problems that arise as a result of distributed teams. An interesting split emerged between those who were establishing distributed teams and those who had established teams. The latter were more concerned with maintaining a consistent vision and architecture having got the practices sorted out to some extent.

There was some debate about models for ensuring a standard architecture. Those backed by successful experiences were:

Seed team - having a small team produce the load bearing walls and I0 and then move each member of the team to one of the distributed locations to be 'keeper of the flame'. The 'keeper of the flame' then needs to hand off to a local member of the team once the appropriate degree of indoctrination has been performed.

Lead team - one development team owns the architecture and there is one chief architect where the authority lies. Expect some international travel.

Black boxing - have off shore teams building to a specification, treat them like a 3rd party vendor, don't care about their process or architecture, just make sure the tests pass.

Beach head - approx 80% off shore, 20% home base.

There was debate about whether distributed teams should work on the same codebase or whether distributed teams works better in a component based environment. People were using a mixture.

I made the point fairly strongly that the model to be used depends on what is to be built. If it is technological activities such as transformations for integration then Black boxing works. If it's business level functionality then one of the other models is more appropriate. I'm also more in favour of Seed team and Beach head for business apps.

Clearly there is room for guidance here and I believe the work myself and a colleague have been doing in this area may well yield some benefits over the coming year.

Minimalism and signs

2005-04-24T17:43:00.000+01:00

In the workshop at SPA 2005 on beauty[1] I was thinking about clarity and minimalism. I value doing something one way. That is, if you are going to introduce a construct, you do it in a consistent manner. I think this is important because someone working on the code is building a model of the system in their mind and consistency helps. If, when they see a certain pattern of implementation, they can always map it to the same intent, they have less to learn than if the same thing is done in different ways in different places in the code.

What is happening (perhaps) is that the code authors are building a shared (and private) language of signs. Since signifiers are somewhat arbitrary and cultural specific it is certain that a team of people will come to the project with different sign systems. To ensure clarity of communication and a shallow learning curve, the team need first to use signs that the team have a shared common understanding of. Secondly, the team have to evolve their own sign system that all buy into. Finally, the authors need to make sure the same signifier is used to represent the same concept throughout the code. When authors don't do this, teams always complain of inconsistency. Worse still, team members may attribute their own concept to the signifier and make incorrect assumptions. This line of thought leads me to think that minimalism and clarity are concerned with creating a system with a minimum set of signs (at some level of abstraction) where one concept is not represented in the code by multiple signifiers. Furthermore, to ensure clarity, the signs should be understood by the team and by the community from which maintainers will be drawn. This in turn infers that the software becomes targetted at a specific design or implementation school of thought.

In conclusion, all things being equal, it makes sense for a team to use a sign they all understand. It maybe that the technology underlying the signified concept is the same, but if the team do not have an understanding of the signifier, confusion and defects await.

ACCU 2005 - Small Memory Patterns

2005-04-24T17:30:00.000+01:00

This talk provided 3 patterns from Charles Wier and James Noble's patterns book on programming on small memory machines. The talk covered the following patterns; Memory Limit, Packed Data and Fixed Allocation. They were pretty straightforward:

MemoryLimit - limit the allocation of memory and fail requests over that limit. The contacts list in a phone is an example of this. Charles also talked about other strategies such as FIFO and LIFO that are often used to decide whether to accept a request but do so by throwing away an existing allocation.
Packed Data - bit packing data.
Fixed Allocation - allocating all the memory you need for an operation ahead of time. For example, 911 systems in the US have memory pre-allocated so that a 911 call will not fail due to an out of memory error.

These techniques are commonplace and well recognised. The talk then went on to discuss techniques the audience have used. These ranged from using multiple heaps to avoid heap fragmentation through to using negotiation between devices to ensure there was capacity to honour a request and if not to tell the requestor to ask for less. The summary is a good place to look for outlines of the wide variety of approaches found in the book.

When considering the applicability of this book it's worth noting that small memory systems are not only systems with a small amount of memory (like a watch of a phone) but also systems with large numbers of users where each concurrent request ends up having a small amount of memory.

Whilst listening to this I wondered what a book on patterns for programming in 'low power' environments would look like. A book on low power environments would contain patterns that optimise on clock cycles, that discusses off-loading computation to hardware and would probably be very useful to JavaScript programmers developing in IE. I've been pondering IE JavaScript patterns for a while and it seems like many of the approaches found in embedded systems development[1] have and can be applied to the IE environment. The reason for this is two-fold:

Performance. There are many activities in IE that have unexpected and significant performance hits. The developer spends a lot of time optimising JavaScript web applications in IE when there are reasonable quantities of data on screen.

Memory. IE, its' DOM and associated clutter consumes a substantial amount of memory. Developers have to minimise memory consumption to get a web app to perform.

----
[1] Of course, programming embedded systems now is not dissimilar to programming non-embedded systems 15-20 years ago.

ACCU - C++0x

2005-04-24T16:55:00.000+01:00

At certain points in my career I've ended up doing alot of C++. Often I wrote a template code, many times in the generative style. I learnt a few lessons from doing this. Firstly, the error messages are terrible. The key reason for this is that the compiler needs all the code present to ensure that the parameter types passed in to the template support the operations that the template code relies on. Hence, the compiler (effectively) expands the code, and only when deep in the bowels of the template code does it report back 'hey, type T doesn't support T::value_type'. When it does that the error trace is enormous, and without a good understanding of the templated code it can be hard to make sense of the error. The second reason was the hoops you had to jump through to ensure that it was clear what type of T you wanted passed in to a template. This is where traits came in, verbose and clunky, but they did the trick, kind of.

All that is going to change - I'd say 'about to change' but the C++0x standard is now scheduled for 2009. C++0x is introducing some features to address the issues raised above, and others. Stroustrup spoke of these at ACCU 2005. Broadly speaking he targetted two areas. Firstly, auto and declspec relate to querying the type information.

e.g.

int doSomething () { };
auto value = doSomething (); // value is an int.

decltype returns a type, e.g. decltype(doSomething) is an int. Syntatically decltype is considered as a typedef name. This allows easy implementation of generic forwarding:


template <class T> auto forwader (T &t) -> decltype (doSomething(t)) {
 return doSomething (t);
}

Secondly there is a lot of work being done on improving the checking when using templates, specifically around traits and a desire to seperate the template code from its definition. The traits idiom has been used in the past to:

Provide specialisations based on the characteristics of a type (i.e. it is a forward iterator).
Explicitly say 'this implementation only works with types with this characteristic'.

C++0x is introducing concepts. The form is yet to be agreed in detail, though there are a number of design ideas on the table. The general idea is that a concept is a set of predicates that assert facts about the sort of type that would work with a specific template. This means that a developer can say 'this template requires type T support ++, assignment, random access' etc. However, it goes further than this and allows new concept to be introduced to represent abstract notions that are specific to a domain, e.g. 'works for small types' or 'works for large types', where the definition of small and large is in the developers mind.

The code may look like this:


template <typeid X>
concept some_trait {
};
template<class T> someClass where {some_trait<T>} /* class def follows */

The {} after the where clause is a set of predicates requiring specific concepts be supported by the class T. Of course, we have to define how a concept is defined in terms of valid types and operations. This could be like this:


template <typeid X>
concept some_trait {
 typename X::iterator; 
 X operator++(X);
};

So, this means the class argument to someClass must support the increment operator and have a type in its namespace named iterator.

Very useful indeed.

Further information is available on the open-std.org site and at J Siek's publications page.

ACCU - Model Driven Development

2005-04-24T16:50:00.000+01:00

The outstanding thing about this talk was Frank Buschmann spending some time discussing the characteristics of the domains in which MDD worked and discussing thoughts about the costs/benefits. It is unusual in my experience to go to a technology talk where the context within which something should be applied is described so well, it made for a great talk. In summary Frank was saying that MDD could be cost effective in domains that had the following characteristics:
+ Closed.
+ Well understood.
+ Undergoing a low rate of change.
+ Had very clear borders, concepts were either in or out of the domain, there were no grey areas.

It looked like the effort required to build the MDD infrastructure (e.g. generators etc) was high and the cost could only be recouped if the infrastructure could be re-applied frequently without incurring more significant cost rev'ing the infrastructure.

Successful examples given related to situations where a product had many different variants that needed generating. These were often drawn from the automobile domain, where software is configured and assembled based on the target car, engine and associated gadgetry (e.g. aircon). Other domains used in the examples included technological areas such as error handling. An interesting side effect of this approach related to coupling. The models had components with low coupling but it was noted the output was often tightly coupled. Since the developers were operating at the model level this tight coupling wasn't important.

It felt like MDD was really tied to product line architectures and not so useful in a service company that works in many different domains and build many different (rather than similar) applications. Having said that, some service companies build a lot of web based CRUD/Workflow apps so there will be mileage there.

As an aside; several conversations were overheard of the form 'generated code isn't elegant, it's useless'. I'm not sure elegance is important. After all, do we care whether the assembler generated by compilers is elegant?. Generated code needs to be efficient, but unless a developer is expected to regularly interact with the generated code (e.g. editing/debugging), it doesn't have to have many other characteristics.

SPA 2005 Resources

2005-04-11T13:03:00.000+01:00

Papers and slides

You can find project approaches and case studies at the BJSS website.

Falling Blue - Agnes Martin

2005-04-04T12:02:00.000+01:00

Horizontal pencil lines perhaps half an inch apart are drawn to the edges of a border of bare canvas. In between the lines long horizontal strokes of dark blue are painted with a small brush from one side to the other. Each line starts, fades and continues as the brush ran out of paint and was then re-loaded. Whilst the painting exhibits minimalist attributes it hasn't been screen printed or otherwise mechanically produced. Each part has been meticulously painted or drawn. It reminded me of the engineering/craft debate in software. External observers view software as something constructed from hard edged parts. Graphical notations used emphasise the mechnistic nature through hard edge iconography. Compare that with the code itself, where each part has been individually crafted, each line tabbed in and placed in relation to the lines around it. In so many ways software felt like Falling Blue.

ST 2005 Note #7 - Further Reading

2005-03-15T08:20:00.000Z

W3C Semantic Web page.

Sem Web Central - open source tools.

Mindswap. An academic group, the director of which co-authored the original Semantic Web paper with Tim Berners-Lee et al.

IBM Alphaworks semantic technologies page.

DAML Semantic Web Services.

Semantic Web community portal.

ST 2005 Note #6 - Using semantic technologies today

2005-03-15T08:19:00.001Z

Current state

The semantic web is here and the supporting standards; XML, RDF and OWL are current W3C recommendations. Vendors and the open source community are providing tool support for ontologies. Some examples are:

Open Source
- Jena 2 - HP's semantic web framework.
- SWOOP - An ontology editor from mindswap.
- Protege - An ontology editor from Stanford. Comes with an OWL plug-in (amongst other things).
  SWeDE is a semantic web development environment built on the Eclipse IDE.
Commercial

Unicorn have a commercial semantic offering they target as an entire architectural solution.
SchemaLogic provide tools for enterprise vocabulary and taxonomy management.
Network Inference provide a semantic web offering and ontology management.

Certainly semantic technology has yet to cross the chasm, but Early Adopters are obtaining value from the application of the new technologies and existing large corporations (both private and government) have been using elements of the semantic conceptual stack for many years.

from www.writersblock.ca/ summer1998/bookrev.htm

Key Lessons

Experience reports from ST 2005 suggest the following lessons:

Starting Points

Start small and focus on a specific business subject area.
Roll out a vocabulary or taxonomy first and allow this to become bedded in. In particular, this approach does not require RDF or OWL adoption.
Drive the semantic effort from EAI/EII initiatives to get good traction.

Approaches

ST 2005 practitioners are of the opinion that federated ontologies are good practice for large organisations. Organisations need to accept that they will end up with multiple overlapping ontologies. Provide high level upper ontologies to map these overlapping ontologies to and/or provide infrastructure for mapping between ontologies.
Define vocabularies, taxonomies and ontologies from both top down and bottom up perspectives. It is important that a top down modeling activity provides broad shape to semantics work. However, a bottom up approach is required to make sure the semantic activities capture information as it is today. Defining a 'to-be' model without understanding the existing semantics of systems is not recommended by practitioners met at ST 2005.
Examine the producing and consuming systems and ensure that the semantics of the existing models are captured in enough detail to validate any vocabulary work.
Avoid attempting to build an enterprise ontology from the ground up. These tend to fail due to the time it take to build them and the federated nature of large organisations. Use integration driven vocabularies to drive out elements of the Enterprise data model piece by piece. Only the concepts shared between applications are in the model, but it is a significant improvement and it's an improvement that can happen in relatively simple stages.
Leverage existing ontologies (such as those from XML standards bodies like OASIS).

Governance
- A vocabulary, taxonomy or ontology will need well defined governance and, in large organisations, a lightweight and nimble standards group to maintain the integrity of the system.
- The UPS speaker (had a metadata repository that started in the 80's) recommend having a QA group responsible for the integrity of the models.
- Control naming of data elements so they are inline with the vocabulary in the organisation. This allows schema (database or XML) to be reconciled back to the vocabulary.

ST 2005 Note #5 - Semantic Web Services

2005-03-15T08:19:00.000Z

Web Service proponents know that the web has only grown so effectively because it allows individuals to establish their own web sites, date repositories, models, APIs etc. W3C standards are really substrates that allow individual expressions to be integrated at some level.

A Web Service world where every service in a subject area uses the same data model is unrealistic and does not reflect experience of the web to date. Web Service proponents need to find mechanisms to map between web services' different models of their domain.

UDDI is a current standard for service discovery. However, to query a UDDI registry the requestor must submit a query in the implicit ontology of the UDDI. What is actually required is the ability for a service provider to describe the capabilities of the service in one or more defined ontologies. A service requester should be able to search for a service using capabilities defined in one or more ontologies. Providing the discovery service has mappings between these ontologies then it will be able to provide matches to searches that may otherwise have returned no results.

Describing a web service using an ontology will not only aid searching but it will also aid use. For a requestor to use a service without additional coding it will need the service to specify the ontology for the terms used in the WSDL. In addition, a requester needs to understand pre and post-conditions for a service and the effects of a process or service. For example, if ordering a book from Amazon it would be helpful if a service requester could determine if an order would result in a book being dispatched or whether other process steps are needed. In the Semantic Web Services world OWL-S is attempting to provide some aspects of this functionality. A service can describe itself using OWL-S and document:

Pre and post-conditions.
Effects of operations.
The semantics of the terms used in the WSDL.
Capabilities of the service.

With these facilities, OWL-S is providing foundations for semantic discovery, automated web service composition and the ability for a requester to determine what the inputs and outputs mean. Further information can be found in this paper.

ST 2005 Note #4 - The Semantic Conceptual Stack

2005-03-15T08:18:00.000Z

The Stack

The conceptual stack in the semantic technology arena is composed of the following key elements:

Layer	Description
Syntax	The underlying representation of the structure. XML provides this foundation piece.
Vocabulary	A collection of terms and their definitions.
Taxonomy	A collection of terms organised into a classification scheme.
Ontology	A specification of a conceptualisation[1]. More practically, data, structure, meaning and rules.

As an organisation moves up the stack the degree of detailed and coverage of the semantic content of the business grows. This in turn means that the utility of the models grow. However, the effort required to define and manage the models increases rapidly.

Decomposing the stack

Each layer in the stack can be decomposed into high level pieces. One such decomposition is below:

This figure shows the constituents of each layer related to semantics.

Vocabulary

The first rung on the ladder, once a common syntax has been defined, is the vocabulary. At its simplest this contains terms (e.g. Currency) and a definition (e.g. medium of exchange, monetary system). This can be elaborated in a number of ways. Firstly the terms can be mapped to an existing lexical database such as WordNet. In this manner the definition of the term Currency could simply be http://wordnet.princeton.edu/cgi-bin/webwn2.0?stage=2&word=currency&posnumber=1&
searchtypenumber=2&senses=1&showglosses=1. Secondly, the term could be defined in relation so surrounding terms. In particular guidance on how to rule whether an item is a Currency is very useful. The medical profession uses this mechanism (rule-in/rule-out) to provide mechanisms for determing if symptoms rule in or out a specific disease. Finally, a term maybe associated with a canonical name and a short name (for use in database schema etc).

Taxonomy

The second rung, a taxonomy presents terms within a classification framework. A taxonomy would classify terms. For example, Currency could be a unit of measure, it could be countable.

Ontology

The third rung, an ontology takes terms, defines the data associated with them, the relationships between terms and constraints/rules that define how terms and relationships can be combined and what their lifecycles are. An ontology can be viewed as a data model with an associated constraint language or as a sequence of assertions of the form [subject, predicate, object].

Practical application

Elements of this stack are in use now in many organisations.

Metadata repository

For instance, UPS has a metadata repository (started in the late 80's) which stores a Taxonomy. Terms are associated with a definition, a canonical name, a short name and an abbreviation. UPS ensure that all schema that reference terms use the standard names. This makes it relatively simple to understand the meaning behind the entity definitions in, for example, a database schema. UPS use the classification scheme to reason about the vocabulary. For example, one classification is code. This allowed UPS to identify they they had a growing list of code terms and move to establish a central code repository and identify sources for codes (such as standards organisations).

B2B standards alignment

Other organisations are establishing defined ontologies within specific domains. This has allowed groups to map an internal ontology to external standards. This activity has enabled standards alignment for B2B activities (both internally and externally).

Web Services

There is a lot of activity around semantics and web services. This is covered in ST 2005 Note#5 - Semantic Web Services.

Enterprise Application and Information Integration

Organisations use a common message bus and/or data bus[2] to make the integration activity cost effective in the medium term. Implicitly or explicitly, the data on these buses normally has a common data model and master/static/reference data source. Without these elements the bus becomes a conduit (an expensive one at that) for point to point interfaces. When a common bus is used it is important that the common models in use can be communicated clearly and all involved understand the semantics of the model. This degree of understanding involves the following elements in the model:

Entities
Relationships, including roles, cardinality, ownership etc.
The business meaning of each entitity.
The business meaning of each relationship.
The lifecycle of the entities and relationships.

It is this information that ontologies document and the current crop of W3C standards provides mechanisms for persisting this information in a machine readable form.

Inference

Ontologies provide the opportunity for organisations to infer implicit relationships between instances based on the explicit relationships in the ontology and associated business rules. However, there was limited example of its used on commercial organisations at ST 2005.

Summary

Organisations are already using elements of the semantic conceptual stack.
Vocabularies, taxonomies and ontologies are reducing the cost of information and application integration.
The entire stack does not have to be adopted at once.

[1] Gruber, Tom. http://www-ksl.stanford.edu/kst/what-is-an-ontology.html
[2] Data bus could be a data warehouse or an operational data store.

ST 2005 Note #3 - The Semantic Tech Stack

2005-03-15T08:16:00.000Z

The semantic web has a technological stack that looks like this (from a W3C perspective):


+---------------------------------+
+                XML              +
+---------------------------------+
+                RDF              +          
+---------------------------------+
+            RDF-Schema           +
+---------------------------------+
+  Ontology Vocabulary (e.g. OWL) +
+---------------------------------+
+              Logic              +
+---------------------------------+

RDF and RDF Schema have been covered in brief in earlier notes. OWL is based heavily on work done as part of DAML+OIL. The aim of DAML was to provide the ability to infer facts from an instance document and its associated ontological schema. OWLs aim appears to be very similar. As we move up the stack we find increasing capabilities in a number of areas:

Ability to define types, relationships and constraints (i.e. structural constraints).
Ability to infer facts not explicitly represented in an instance document.
Ability to define semantic constraints, business rules in other words[1].

This comes at a price; ontology production[2], validation costs rise and inference engine sophistication rises.

[1] Not sure this is different from 1) in any meaningful way but it feels different.
[2] As more sophisticated ontologies are modeled using complex logic assertions then more analysis and design time is required for each ontology.

ST 2005 Note #2

2005-03-14T21:28:00.000Z

The problem with the RDF in Note #1 was that we could not constrain the RFD in anyway. A publisher of a movement could add any triples it liked. For many purposes this is not enormously useful.

The last RDF snippet defined was this:


<types:Movement rdf:about="http://www.newco.com/movements/1234">
   <terms:carries rdf:resource="http://www.newco.com/grades/5678">
</types:Movement>

<types:Grade rdf:about="http://www.newco.com/grades/5678">>
   <terms:name>JET</term:name>
</types:Grade>

What is needed is to ensure that Movement only has one property, carries, and that it can only map to instances of Grade. Imagine we are creating a schema at http://www.newco.com/movements. We'll define each class as an ID within that schema, i.e. a Movement will have URI http://www.newco.com/movements#ID. We also need the namespace for RDF schema.

This means we need to change the namespace of our instance document, and we'll want to reference these namespaces within content so we'll define ENTITY elements as well:


<!DOCTYPE rdf:RDF [
  <!ENTITY rdf 'http://www.w3.org/1999/02/22-rdf-syntax-ns#'>
  <!ENTITY movements 'http://www.newco.com/movements#'>
  <!ENTITY rdfs 'http://www.w3.org/2000/01/rdf-schema#'>
]>
<rdf:RDF xmlns:rdf="&rdf;"
  xmlns:movements="&movements;"
  xmlns:rdfs="&rdfs;">

We've changed the types namespace so it references IDs - note the trailing #.


         xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
         xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"

... and the XSD namespace if we reference XSD types.

Anyway, back to the schema definition. Define Movement as a class:


<rdfs:Class rdf:about="&movements;Movement"
  rdfs:label="Movement">
 <rdfs:subClassOf rdf:resource="&rdfs;Class"/>
</rdfs:Class>

Likewise, let's declare a property carries, ensuring it maps to a grade and is associated with a Movement.


<rdf:Property rdf:about="&movements;carries"
  rdfs:comment="Carries relationship mapped to Grade"
  rdfs:label="carries">
 <rdfs:range rdf:resource="&movements;Grade"/>
 <rdfs:domain rdf:resource="&movements;Movement"/>
</rdf:Property>

but we need to ensure Grade is defined:


<rdfs:Class rdf:about="&movements;Grade"
  rdfs:label="Grade">
 <rdfs:subClassOf rdf:resource="&rdfs;Class"/>
</rdfs:Class>

Now let us define a name property for Grade:


<rdf:Property rdf:about="&movements;name" rdfs:label="carries">
    <rdfs:domain rdf:resource="&movements;Grade"/>
    <rdfs:range rdf:resource="&xsd;string"/>
</rdf:Property>

We don't have to, but we can also make the fact xsd:string is a datatype explicit thus:


<rdfs:Datatype rdf:about="&xsd;string"/>

One particularly interesting facet is that the properties are defined outside of classes. They are then associated with classes using the rdfs:domain attribute.

However, as the W3C primer notes, RDF doesn't address:

cardinality constraints
specifying whether a property is transitive
specifiying that a property is a unique identifier
specifying that two different classes (different URIs) represent the same class., ditto instances.
disjoint classes
class specific range/cardinality constraints

This is where OWL and other richer schemas come in.

ST 2005 Note #1

2005-03-14T21:26:00.000Z

A number of organisations have been discussing how they are using RDF as their data interchange format. They describe their use of RDF and present examples of RDF encoded in XML.

When representing data in XML it often boils down to how you use XML to represent a directed graph, where the arcs are labeled and have meaning. This in turn raises the value/identity issue. For example:


<movements>
   <movement>
         <grade><name>JET</name></grade>
   <movement>
   <movement>
         <grade><name>JET</name></grade>
   <movement>
<movements>

In this example we assume that the message has to be self-contained, i.e. no external references. The example includes the grade JET, which is identified by value. There are a number of things we don't know. What is the relationship between a movement and the grade. Does a movement carry the grade? If a movement is deleted is the grade deleted as well? Addressing the first issue involves making the relationship a first class element. Whilst doing that let's ensure there is only every one JET grade defined in the document.


<movements>
   <movement>
      <carries>
         <grade id='1'><name>JET</name></grade>
      </carries>
   <movement>
   <movement>
      <carries href='#1'>
         <grade><name>JET</name></grade>
      </carries>
   <movement>
<movements>

In this example the second carries back references the first one. The implication is that the JET grade has an identity which is important. In complex schema this type of approach is often used, though the references and the referenced may have different locations. Of course, there are issues around containment (i.e. when I delete the first movement element does that delete the JET grade or not. However, let's ignore that for the time being.

Note that an approach to representing relationships has been constructed 'on the fly'. It's not standard and the semantics aren't clear.

We could express the each movement as a set of triples (subject, predicate, object) thus:


movement (some id) carries grade (some id)

Or, with a liberal addition of URIs for identifiers:


http://www.newco.com/movements/1234 http://www.newco.com/predicates/carries
                                              http://www.newco.com/grades/5678.
http://www.newco.com/grades/5678 http://www.newco.com/predicates/name JET.

This URI based triples model is what RDF uses. Moving to XML, or an XML representation of the triples, we'll assume the following pre-amble:


<?xml version="1.0"?>
<!DOCTYPE rdf:RDF [<!ENTITY xsd "http://www.w3.org/2001/XMLSchema#">]>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
         xmlns:terms="http://www.newco.com/terms/"
         xmlns:grades="http://www.newco.com/grades/"
         xmlns:types="http://www.newco.com/types/">

In RDF we could describe the movement and grade thus:


<rdf:Description rdf:about="http://www.newco.com/movements/1234">
   <rdf:type rdf:resource="http://www.newco.com/types/Movement"/>
   <terms:carries rdf:resource="http://www.newco.com/grades/5678">
</rdf:Description>

<rdf:Description rdf:about="http://www.newco.com/grades/5678">
   <rdf:type rdf:resource="http://www.newco.com/types/Grade"/>
   <terms:name>JET</terms:name>
</rdf:Description>

Note we added type, so we know what type of this we are dealing with. For brevity, we can remove the rdf:Description and rdf:type verbosity by using the type as an element name:


<types:Movement rdf:about="http://www.newco.com/movements/1234">
   <terms:carries rdf:resource="http://www.newco.com/grades/5678">
</types:Movement>

<types:Grade rdf:about="http://www.newco.com/grades/5678">>
   <terms:name>JET</term:name>
</types:Grade>

What have we gained over the initial XML?

We have a formal model for representing information about an entity (triples) which we do not have to invent.
We have a well defined mapping from this model to XML and we didn't have to invent it.
We haven't had to invent a mechanism for handling the fact our grade/movement model isn't hierarchical, the types are peers.
We have not ended up in a world of xsi:type pain.
We have RDF tool support if we need it.
If we need collections with defined semantics then RDF supplies these, we do not have to invent them.

Clearly RDF is a big topic, and there is also RDF-Schema and then OWL to layer some rules on top of structure. More on this later.

[1]The ST 2005 Notes are consolidated notes from the 2005 Semantic Technology conference.

Creating space for evolutionary design

2005-02-26T14:07:00.000Z

Fowler seems to be observing what many have been using as an objection to the "let's use evolutionary design for everything". Good technical leadership (as opposed to dictatorship - do not confuse the two) is required for successful software projects, particularly those above a certain size. In one of his examples, a problem where a project had multiple solutions to the same problem was fixed by adding an 'architecture team that forged a base approach to these problems'. He then goes on to say that in fact he thinks it is "Continuous attention to technical excellence and good design enhances agility." that is needed. Well yes, and that requires the team need to be able to spot better solutions or identify unwanted divergence[1] in the existing solution. That's hard across a code base where your team can't keep it all in their heads or where the collective understanding is spread across too many teams to make a query efficient.

In larger projects that often means you need a team that addresses the macro structure. They are needed to ensure that the project establishes and looks after the load bearing walls. After all, on larger projects it is these load bearing walls that end up delineating the space within which evolutionary design can occur.

[1] sometimes you want divergence, because the original solution has reached its limits.

Turning a plan inside out

2005-02-24T18:43:00.000Z

Big plan, no idea whether it delivers what is needed, no mappings back to original stuff that needed to be done.

The (successful) approach we've taken is to turn the plan inside out thus:

Pass 1:

Each team to list every deliverable from the 'plan backwards' activity as a 0 day milestone.
Each team to connect each milestone back to the tasks which deliver it.
Each team to create tasks for the milestones that have no dependencies.

Pass 2:

Each team to disconnect dependencies on other teams tasks and reconnect them to the other teams deliverable milestones.
Each team to list deliverables they expected and didn't appear.
Each team to weave new set of deliverables, and therefore tasks, back into plan.

Pass 3:

Each team to remove its teams tasks from the plan, leaving only milestones and create their own workplan from the tasks.

Result

Management can manage by milestone.
Each team can manage by input/output milestone and change their plan as they see fit (modulo not moving agreed milestones)

Transparency

2005-02-24T15:55:00.000Z

ss is a proponent of transparent projects, those where the management can see exactly what is going on. it has a number of advantages:

less management intervention as they can see what is going on without having to poke it.

it builds trust.

it encourages openness.

however, it has at least one drawback. imagine an outsource type situation. if the outsourcing (sponsor) organisation has a large and complex set of processes and metrics, and the outsource organisation does not, then the sponsor is likely to consider its own processes and metrics to be crucial elements in establishing transparency. the sponsor will wish to drive these elements and other forms of governance into the outsource organisation.

if the outsource organisation is lightweight and agile then the transparency can have a negative effect. in the agile organisation all the joints are a little loose, there is room for manouvre, a little chaos.

as the sponsor organisation drives its capital M methods and metrics into the outsource organisation it slowly removes the degrees of freedom in the outsource organisations joints. as the joints stop moving smoothly in many directions the outsource organisation starts to lose its adaptability and ability to innovate. The result will be an outsource organisation that will likely move as slowly or slower than the sponsoring organisation.

to avoid this the sponsoring organisation needs to accept that the outsource organisation is different, and is efficient because it is different. the sponsor needs to turn a blind eye[1] to the micro-level chaos and slack in the joints. it needs to allow the outsource organisation to operate in an effective, adaptable mode, recognising there will be some chaos and flex. to do this it has to ensure that measures and methods it puts in place are macro level indicators, guidance and governance.

this balance is key to successfully outsourcing to significantly smaller and adaptable organisations.

[1] or maybe observe and learn.

Heretics and orthodoxy

2005-02-17T01:59:00.000Z

http://www.ccel.org/ccel/chesterton/heretics.pdf

http://www.svots.edu/Faculty/Thomas-Hopko/Articles/postmodern.html

Orthodoxy, unexamined dogma, maybe a secular pastime in the West now. IT is particularly bad for orthodoxy and codification of guidelines into rules.

http://www.bbc.co.uk/radio4/religion/somethingunderstood.shtml sparked this off.

Extreme Pragmatism

2005-02-17T01:58:00.000Z

Some people have an outlook that is focused entirely on pragmatic action, Extreme Pragmatism. Like everything, pragmatism is good in moderation. Sometimes however it can become too dominant. Often a highly dominant Extreme Pragmatic approach means that deep thinking around approach, drivers and foundations is lacking. This can lead to decisions that do not balance the tactical and strategic views correctly. The highly tactical approach can get quick results, but often at the cost of later regret work. G. K . Chesterton seems to have articulated this issue nicely, though somewhat rhetorically.

Suppose that a great commotion arises in the street about something, let us say a lamp-post, which many influential persons desire to pull down. A grey-clad monk, who is the spirit of the Middle Ages, is approached upon the matter, and begins to say, in the arid manner of the Schoolmen, “Let us first of all consider, my brethren, the value of Light. If Light be in itself good—”. At this point he is somewhat excusably knocked down. All the people make a rush for the lamp-post, the lamp-post is down in ten minutes, and they go about congratulating each other on their unmediaeval practicality. But, as things go on they do not work out so easily. Some people have pulled the lamp-post down because they wanted the electric light; some because they wanted old iron; some because they wanted darkness, because their deeds were evil. Some thought it not enough of a lamp-post, some too much; some acted because they wanted to smash municipal machinery; some because they wanted to smash something. And there is war in the night, no man knowing whom he strikes. So, gradually and inevitably, to-day, to-morrow, or the next day, there comes back the conviction that the monk was right after all, and that all depends on what is the philosophy of Light. Only what we might have discussed under the gas-lamp, we now must discuss in the dark.

I have been working on some XML stuff recently and has spent a bunch of time considering precise definitions of things, an experience I've enjoyed but found hard work. It's a shame there isn't always the luxury of time to do it more often. I believe this groundwork will at least ensure that everyone is on the same page when using their shiny ontological lego bricks. A shared understanding of the foundations behing these bricks must (surely?) contribute to more effective modelling. It could have just defined some basic rules, but it doesn't think that would have worked as well.

GK Chesterton on requirements

2005-02-17T01:48:00.000Z

What is the point of begetting a man until we have settled what is the good of being a man

Russian pavements

2005-02-06T03:23:00.000Z

I went to the photo gallery on Gt Newport Street[1]. By far the best piece was Stories from Russia where the photographer had got the general public to take part in the photo shoots. She has them standing, in front of famous buildings, facing the camera. It's weird, all the people (in the main) are staring at the camera, looking almost like mannequins, and yet real. It's disconcerting. Since she sampled the general public there's a huge mix of different people, defiant trendies, tired old people, the works. Worth a look.
In the back room was a video installation. There is one person on each screen and, individually, they tell their version of the story behind the Hotel Moscow's two designs. First it is told round robin style and a central screen has the translation on it, then it merges and they all speak together and the translations are all merged together, leaving you struggling to pick any words out from the translation, in the same way that you struggle to pick out a single voice in the cacophony of them all talking at once.
Never photographing downwards, I spent some time today photographing downwards on my way into london. I've now got a collection of bits of pavement, tree trunks growing from gravel, railings, barriers and peoples feet in the crowds. I guess I#ve got to figure out what to do with it now but I suspect I'll build it into a sequence, in order of my descent into london.

[1] another weekend of lobbying failed to J of the merits of the V&A.

Initial thoughts on Project Rescue

2005-02-04T09:20:00.000Z

I thinks there is a good parallel between ICU and fixing broken software projects. It's not been thought through very well but here are the, admittedly sparse, notes so far.

Architecture and beauty at the V&A

2005-02-01T13:20:00.000Z

I discovered rather late that V&A have opened an architecture gallery.
Along with the beauty stuff V&A seems to be delivering the goods.

Axis of beauty

2005-01-28T02:05:00.000Z

Whilst beaty must be holistic, it must operate across different axis. For software these might include:

Representation

Textual, how is the text laid out on the page?
Graphical, do the graphical representations hold any aesthetic merit?

Dynamics

The way the program executes, its flow of control.
The way the software is updated as function or added, do new pieces slide into place simply?

Structure

Of the syntax tree.
Of the data structures.

Solution

How elegant is the solution?
How well does it balance the competing constraints?
Output
How attractive is the output from the software?

Visitors versus maps of functors

2005-01-26T16:18:00.000Z

I find that it doesn't often reach for the Visitor pattern. The visitor pattern uses double dispatch, essentially scaffolding to get around the fact that many languages do not have multi-dispatch. In the visitor pattern this scaffolding is an accept (SomeVisitor) method, added for each type of visitor and a visit (ConcreteType) method on the visitor, for each type of object being visited.

I avoid the Visitor pattern because:

I don't like the scaffolding being invasive into the domain object.
I don't like the fact that when it steps through code in the debugger it steps into a method and immediately is bounced out into a corresponding method on one of the arguments passed in.
I don't like the fact the Visitor becomes a huge list of every type in the system, all crammed into a single interface.
If the Visitor needs additional context then the context needs to be added to each accept () method or put in thread local storage.
I don't have the disconcerting debugging experience where you step into a method in a domain object, only find a call out to one of the arguments.

So, instead of double dispatch, I normally rely on a table driven approach where the code looks like this and the ShapeRendererSource maintains a mapping of types to functors:

ShapeRendererSource::renderer (shape).draw ();

The ShapeRendererSource is scaffolding and the registering of functors with the source is also scaffolding. However, the scaffolding is external and adding context is as simple as extending the functor interface.

The approach addresses 1)..5) but has worse performance, creates a set of functors to render rather than a set of methods on a visitor (though to be fair visitors normally delegate out to objects - so you've saved one layer in the table based approach), and means somewhere you have the ugly code:

ShapeRendererSource::addRenderer (Circle.class, new CircleRenderer ());

On balance though I value the lack of intrusion into the domain object strongly enough that I'd choose the table driven approach unless there were performance concerns.

Intentional comnputing

2005-01-19T08:09:00.000Z

"The key components of the proprietary technology are a uniform tree-like representation for all the contributions from all the stakeholders of the software that is produced; and an intentional editor that maintains the invariances that are represented and lets the stakeholders edit the contributions in any number of notations that are projections of the intentional tree."

Isn't that a bit like LISP with better tooling?

Coal mining & software

2005-01-17T17:10:00.000Z

Coal mining was a major technological industry in the industrial age. In addition to technology and innovation it relied on people spending time underground digging.

Software is a major technological industry in the IT age. In addition to technology and innovation it relies on people spending time writing code.

Eventually it became cheaper to pay someone in Poland to dig coal out of the ground than it did to dig the coal out of the ground in some UK mines.

The economic argument that drove once high-tech jobs in coal mining off-shore are now driving software development off-shore. It is an inevitable progression, the question is not whether but rather how do we innovate fast enough to stay ahead of the curve.

Cost versus efficiency

2005-01-17T16:40:00.000Z

Kenwood Corp. made mini-CD players in Malaysia until last year, when Haruo Kawahara, who heads the Japanese electronics company, decided to bring the work back. Now, teams of four Japanese employees do the work that had been done by 22 Malaysians.

I wonder what the equivalent avenues of efficiency improvements are when building business apps in the UK. Maybe there are some low hanging fruit (assuming that you've already got a recruitment policy of 'full stack', 'quality people only' in place):

Knowledge sharing.
Harvesting collateral as it is built and improving in next development.
Having a good sharp set of appropriate tools.
Avoiding NIH at all costs.

Then there is some tougher stuff (assuming that we can utilise the agile stuff in the right way - i.e. short feedback cycles, short defect discovery cycles etc.):

Having a number of foundations; for management, .NET, J2EE (etc) that define some default tools/libraries/approaches that should be used when building 'standard' business apps (as opposed to the exceptions when you *have* to do something different - as the problem demands it).
Reducing defects.
Raising the level of abstraction we use to build certain type of systems. This is the tough one, since it needs an organisation to be building enough of a certain type of app that the abstraction level can be raised.
Canned architectures.
Engaging clients at the problem formulation stage so that the requirements can be distilled down to the smallest set needed for the required business value.
Improving requirement analysis skills (including understanding of language and intepretation) and low cost prototyping so that requirement misunderstandings are less frequent.
Getting the required amount of business engagement, when needed, with the right degree of analysis.
Make 'play' cheap.

Of course, the big problem with this is that in manufacturing (presumably) costs are saved through reducing the cost or time it takes to instantiate a design where costs are saved across many multiple instantiations. In software, it's primarily about reducing the time needed to think hard rather than the instantiation time - since software is normally only instantiated a very small number of times.

However, instead of focusing on the flaws in the parallel consider that what is really going on is a battle of cost versus efficiency and it is about time we joined that battle. In the long run this battle will even out as any efficiency gain made in one area will eventually be equalised or bettered in another geographical area. However, in the short to medium term it may help.

Object Thinking

2005-01-16T11:31:00.000Z

This book by Dave West seems pretty interesting. He did a great talk on magic and software at OOPSLA '03 Onward! track. Proceedings here. A book that actually talks about OO rather than C#, Java, C++, Ruby etc. There are very few of those.

Maybe worth a read.

Probably not a good read if you didn't listen to the monk until you found there was no light to read by ;-)

It became self aware on August 29 1997, 2:14 am Eastern Time

2005-01-03T09:48:00.000Z

Mr Parkinson must have been watching the T franchise while he wrote this.
Alarmist but interesting points. Particularly the piece about building in 'circuit breakers' - I think it's interesting in the context of STP systems as well as 'society infrastructure' software

Unit Testing

2004-12-30T11:17:00.000Z

This statement from http://www.joelonsoftware.com/items/2004/12/06.html irritated me:

However, a good, experienced developer is about 100 times less likely to write bugs that will be uncovered during unit
tests than a beginner. It is therefore practically useless for the former to write these... but most methodologies would
enforce that he has to, or else you don't pass some phase.

Unit tests serve to measure compliance of code with the developers understanding of the requirements and crucially, control project velocity. If defects from unit tests rise then dev time is spent fixing the code, slowing dev down to a pace where an appropriate degree of compliance is achieved.

If you are very experienced and write largely defect free code you may travel fast and light - turning the unit testing dial down to 3 - maybe only testing the really hard stuff. If you are less experienced or generate defects then you may want to turn that dial up to 7. The problem occurs when you cannot guarantee that the code written will always be updated by the same person, or a person with the same defect introduction profile[1]. In this case you may end up with the dial at the wrong setting, defects are introduced and worse, development isn't slowed to a point where the defects stop appearing in the main code base. Of course, you may have tactical additions that don't have the same quality bar as the rest of the code and you maybe able to turn the dial right down, but that's different as a quality bar applies across the team and not to individuals.

So, change the unit test dial for an individual at your peril, you never know when someone else may edit the code.

A wider problem with the statement is that it shows an atomistic approach whereas IT projects and development teams are complex systems and issues relating to those projects and teams need to be considered holistically as well.

[1] Since turning down the dial has to recognise the types of defects you are going to stop testing for.

Rhetoric in IT

2004-12-30T07:30:00.000Z

http://www.joelonsoftware.com/items/2004/12/06.html

I loathe rhetoric, it all ends up so black and white, them and us.

> The trouble with MSF is that it starts with a group of successful developers, who
> are successful because they are resourceful, intelligent, experienced, well-
> meaning, and have plush private offices with doors that close, and then
> attempts to claim that if impose some of their "best practices" on your team of
> unskilled developers, you will achieve the same results. It's like Daniel Boulud
> selling a manual to McDonald's fry cooks. "Out of potatoes? Try Yams. Throw in
> a bit of rosemary. Toss and serve with a lime-basil aioli dipping sauce. Yum."
>It's just Best Practices, right?

I find little to substantiate the claim that MSF says the practices will allow a team of unskilled developers to deliver the same results as successful developers. It maybe that some people, seeing IT and people as a commodity, infer that a process can be applied to a production system irrespective of the people. However, I don't believe that MSF provides the data to support that view.

I would prefer critiques on method in general and in specifics to distinguish between:
a) What was intended and the context it was intended to be applied in.
b) How they are used.
c) The problems in the statements about a) that lead to problematic b)s.

Lo-fi prototyping & Viso

2004-12-16T09:52:00.000Z

Some of you will know that I'm keen on using lo-fi prototyping, or at least the UI flow sketching part, to elicit requirements on user interfaces (static/dynamic aspects) and functionality.

Whilst practising this and trying to sell the technique into an existing project I came across the term wireframing, a term I'd not heard before withint this context. Wireframing refers to the practice of rendering a skeletal GUI and all its navigation paths in a sequence of diagram. This process assists in the process of gathering data, ui and functional requirements. Its relationship with lo-fi prototyping is that lo-fi includes wireframing, but in a lo-tech fashion, i.e. no Visio.

Some people believe you can use HTML to produce your wireframes. I'm not convinced. Wireframing is about diagraming something, preferably with a user. Feels like the most appropriate tool for diagramming would be a diagramming tool. One of the values of using HTML is that the navigation paths can be demonstrated. I imagine this could work for simple user interfaces but be hard to get right for complex ones. I also value being able to draw these diagrams whilst sitting with the user or business expert. I suspect having to diagram in a wysiwig HTML editor would just get in the way and reduce the interactivity.

All in all I prefer to use pencil and paper with a user and then, if really needed, translate that into an editable electronic representation.

Choice

2004-12-09T09:54:00.000Z

"Choice"[1] is a mantra in western society, and everyone wants, is expected to, exercise choice. This is going to drive IT service organisations to provide more and more choice. Choice in risk, delivery model, function, timescales, cost etc.

[1] Choice being distinct from significant variety, knowledge or need.

Commoditised IT

2004-12-09T09:49:00.000Z

IT practioners see themselves as specialists in an area that is hard. Outside the industry IT is seen more and more as a commodity. Alongside this commoditisation we see the rise of pervasive computing, more and more people connected to each other via smaller and smaller devices. Once something becomes pervasive you don't tend to pay attention to it.

This has to effect the industry. There is constant pressure on costs and raging debates about the benefits of a CIO in 'non-IT' businesses along with the dissoloution of IT departments in favour of outsourced teams (badged or otherwise). Why have in-house knowledge, it's a commodity - like buying a hair dryer.

So there is an opportunity - whilst there remains a difference between the perception of IT as a commodity and the actuality[1] there may be an increasing need for specialists who can unpick the mess generated from false impressions of ITs state.

[1] I am entirely unconvinced that IT as a whole exhibits the characteristics of a commodity.

Backward compatibility in XML

2004-12-02T17:57:00.000Z

I wonder if XML messages should contain something like this (when straightforward backward compabiblity by flexible schema is not enough):


<version>
   <number>1.4</number>
   <transform to='1.3' version='0.1'>http://......14to13.xsl</transform>
   <transform to='1.2' version='0.4'>http://......14to12.xsl</transform>
   <transform to='1.1' version='0.3'>http://......14to11.xsl</transform>
   <transform to='1.0' version='0.5'>http://......14to10.xsl</transform>
</version>

so that if you get a message for which you can't process that version, you can
lookup a transform to map to a version you do understand. If there is no transform then its an incompatible version change.

You could even imagine the URL being to a web service to handle further reaching changes, though performance would not be great.

Requirements, analysts and language

2004-12-02T05:34:00.000Z

When you look at new organisations to assist you in COTS based development one of the questions must be 'do they know your business?' and the other 'do they know your domain?'. If they have a product with a good fit then the answers to these questions will be obvious, where there are significant gaps then these questions become more important, or do they? If I'm in foriegn climes then I'll attempt to learn at least some basic of the language, and knowing that mispronunciation can create misunderstandings, will pay careful attention to meaning and pronunciation. When I'm in a country that speaks English I'm careless, assuming that my English is the same English as the reciever of my verbage. This assumption is broken, as anyone who visits the US finds out quickly. It's an assumption not made when trying to speak a foreign language.

I wonder if it is easier to engage with an organisation that doesn't know your domain, since they won't make subtle assumptions about your domain or your business simply because they *think* they know the language you are using.

Maybe 'preferred readings' are easier to construct if you don't speak the same language and don't share words that map to different social constructs.

That's an extreme view but it seems that at least you need to pay more attention to what someone says if *they* think they are speaking the same language as you.

diff transactional data & master data

2004-11-19T09:03:00.000Z

is there any difference?

Master Data (MD) is long lived, so is much Transactional Data (TD).
MD may only be valid for certain subscribing applications, same for TD.
It can make sense to publish MD even when it is not fully populated, same for TD.
MD has both global entities and local entities, that can be the case for TD.
MD often goes through workflow for updates/creation, so does TD.
MD entities are heavily references, TD not as much.
MD needs to be flexible to accommodate significant change, TD less so, since TD is normally consumed by fewer applications.

So if there is a difference that influences design it is probably MDs need to support a higher rate of change than TD and a higher number of external, incoming references.

Components of enterprise architecture

2004-11-19T08:50:00.000Z

architectural principles.
reference architectures
as-is/to-be business process maps.
as-is/to-be organisation/roles.
as-is/to-be data and process flows through systems (from actors through to end nodes).
as-is/to-be functionality (inc. requirements for to-be world)
as-is/to-be system landscape.

as-is/to-be technology landscape.
as-is/to-be infrastructure architecture.

Next layer down:

Domain Model (static, dynamic)
Business Rules
Software Architecture
Infrastructure Architecture
Integration Architecture
Application Architecture (functions mapped against system)
Deployment Model
Workflow Specification

Layers below this are construction type activities. First punt, I'm sure there are more elements. I guess we are answering who, what, where, when, how. Of course Zachmann would suggest the above are broken down into 36 cells of normalised deliverables. I'm not sure I buy this type of model yet.

Entities more important than relationships - huh?

2004-11-11T18:14:00.000Z

Why do the popular languages not have associations as first class entities?
Why do many modelers not see relationships as first class entities worthy of naming and attributes?

What is it about associations that encourages this wanton neglect?

I want to know.

Container elements

2004-11-11T14:23:00.000Z

If an entity of type A has an association C to many entities of type B the XML could be written thus:


<A>
   <B>
   <B>
   <B>
   ....
</A>

but there could be multiple associations to entities of type B so maybe it needs to be:


<A>
   <C>
   <C>
   <C>
   ....
</A>

but now the element name representing entity type B is polluted with the association name. Perhaps it should be like this:


<A>
   <C>
      <B>
      <B>
      <B>
      ....
   </C>
</A>

advantages of this are that the association is a first class element, so it can have attributes and the name of the association doesn't effect the name of the contained elements. This also means that role names on associations (in UML) have a precise mapping to an element in the XSD.

I don't think this applies to unary associations though.

Practically it also seems to me that the traversal /A/C/B expresses the intent better that /A/C.

Dom Sub COTS

2004-11-09T09:20:00.000Z

When building a programme around COTS do you go for a submissive or dominant partner? Who will do what you want and who will force you to control your desires? Which one of those options will work better? What signal will you agree? Do you want short deliveries that don't quite meet your expectations or do you want to be waiting and waiting and waiting.....?

The trouble with concensus

2004-11-04T04:23:00.000Z

Consensus, stakeholder engagement, buy-in, continued business sponsorship, standards followed, fewer obstacles.
Design by committee, less effective decision making, 'all things to all men', group think, group polarisation, slow, distrust.

'grist to the mill', making it happen, get things done, more effective decision making, speed, trust, benevolent dicator.
No buy in, constant debate, dictat, favoured few, command & control, more obstacles, internal & external resistance.

Consensus requires engagement, engagement requires time, both from within and without. Consensus within large teams involves many people. Involving many people means many people are involved in many things.

Apartment thread the project and consensus is reduced. Small teams reach conclusions that are communicated but many people aren't involved in many things. The project is simpler to understand and more transparent. Trust is required, hard but once it's established is clearly beneficial. Each work packet has dedicated resource and energy is focused in one place. The project is invigorated and dynamic.

Everyone will attest to the fact that having someone working on multiple activities at the same time creates problems when tracking work or ensuring work gets done. However, it wasn't until this week that I realised how much consensus works to break down singularity of purpose. Consensus doesn't just make reaching decisions harder, it de-focuses the team. Worse, it's a slow acting poison. Initially reaching consensus makes life easier and more efficient. As the team grows the drip drip drip of consensus sets expectations. Before you know it, all the key players are on all the work streams, so they can all reach consensus.

I think it was really only this week that I realised just how significantly consensus de-focuses individuals.

Caveat

Consensus on small projects is easy, there is only a small group of people. My comments only apply to large projects where people end up being involved in N strands of work, just to ensure that consensus can be achieved.

Hyperreality in LiveJournal

2004-10-22T23:36:00.000+01:00

Baudrillard used the term hyperrealism to mean a world where reality is represented as more perfect than reality (crudely speaking). I've been watching the 'latest posts' section on LiveJournal and, when I read them, things always sound more interesting than they probably really are. This could be for any number of reasons; the mundanity of every day life is edited out, there can be a tendency to mildy exaggerate for comedy value etc . I don't mean to make a judgement here, there isn't anything wrong with these things, an LJ blog detailing the daily grind would be dull dull dull.

Umberto Eco says (writing about Disney Land) 'Once the ‘total fake’ is admitted, to be enjoyed it must be seen as totally real'. Well I suspect LJ readers quite enjoy the ride.

Maybe the LJ community is an example of hyperreality . Like a visit to Disneyland, we all suspend our disbelief and 'go along with the ride' to get maximum pleasure from the experience.

Customisation and products

2004-10-07T23:27:00.000+01:00

I used to work for a low volume high value product house (in fact I have worked for two of those). Low volume high value software are always being modified. Small companies are selling these products to large organisations. These big organisations always want special changes, sometimes because they really need it, and sometimes because it establishes a power relationship. Modification requests can sometimes be an assertion of a power relationship, it makes a point.

As a result, successful niche products often have meta data driven architectures. Attributes can be added, forms can be re-designed and business logic added. I've always enjoyed writing these types of architecture, they are hard to get right and it is often challenging to make them perform.

Compare this with bespoke or high volume systems where it's about getting a fixed solution out there at minimal cost. Since maintenance and enhancement budgets for bespoke systems often comes out of seperate cost codes, the argument for building a flexible system is impossible to make. This despite the fact that the only constant in todays world is change.

What the IT world needs is a fast, efficient meta-data driven architecture for business apps. Whilst there are a few around, most either run like slugs or are too restrictive. However, I've seen plenty of business apps now, and surely delivering an architecture that supports the creation of these is achievable. There was a book written by Pete Eeles (and someone else) called something like Business Objects, it was written in the mid-90s and proposed a model that influenced SS when it wrote a COM based architecture. However, it (the 'business objects' model) just never seemed to provide the right degree of customisation. Customers want to build systems out of big lego bricks, but they don't want them to look like they were built from big lego bricks. *That* is the challenge.

If you know of any good meta data driven architectures that are efficient, scalable, reliable and unrestrictive then make a comment.

Play

2004-10-05T20:41:00.000+01:00

I've been reading some book that describes the history of scientific discovery (books aim being to debunk scientific method) and comparing the scientific method with the actuality of the creation of ideas and construction of application or experimentation. What seems pretty clear is that this process is akin to play, i.e. the idea and construction grow in an intertwined fashion.

Lots of people are very keen on interpreted languages, particularly those like Ruby, Smalltalk, Tcl etc. The reason for this maybe that it allows the developer to play and maybe play is very effective and efficient. The problem comes with creating a safe play environment on large systems. Maybe if someone cracks that problem I'll get a more rewarding environment in which to build stuff.

Oh, and we'd get to use playdough to produce systems.

Boundaries

2004-10-04T21:54:00.000+01:00

People like to draw boundaries. A boundary enables the description of elements that lie within and without the boundary. Drawing boundaries also has other effects:

Boundaries can frame thoughts and hence lead to consideration of the inner or the outer without considering the whole. (are you with us or against us?)

Boundaries allow you to characterise the nature of the boundary, i.e. permeable, semi-permeable, flexible, fixed, etc.

Brian Maricks recent exposition on testing metaphors includes a boundary, that between the Agile team and the rest of the world. He uses the following image

and says:
They - programmers, testers, business experts - are in the business of protecting and nurturing the growing work until it's ready to face the world.

I thinks the choice of boundary is interesting, Brian's choice is large, solid, horned defenders of the growing work. This team looks independent, defends its young and I imagine it would survive outside of its current host organisation. That doesn't sound like the software projects I'm familiar with. In my experience (which includes software consultancy/service companies and niche product houses), projects tend to not have an existence outside their host organisation. Occasionally teams move to a new host, but it is uncommon. I don't think Brian intends for the team to be independant from its surroundings, but I can see how people may interpret it in this way. This is the problem with drawing boundaries and not discussing the nature of the boundary.

Maybe the testing metaphor (which is really a team metaphor) needs to acknowledge the interconnectedness of the team with the host organisation. I think the metaphor needs to conflate team identity, interconnectedness and symbiosis (though it has to be admitted, some projects are parasites that kill the host). Maybe neurons work as a metaphor, they have a strong centre, they are connected via fine tendrils into the host, communications down particular links reinforces communication paths and they work to form a whole that is greater than the sum of the parts. Initially attracted to that idea, I don't see it as very successful. Brian's desire seems to be to bring independance, critical thought and error seeking behavior inside rather than outside the team is laudable, and isn't addressed. However, the dinosaur ring metaphor chosen is akin to protecting the child against all comers, seeking to nurture it but not allowing it outside until it is grown. Is this really the right way? Shouldn't the child be encouraged into the open early, supported but not closeted, exposed to new people and ideas and encouraged to explore? Social networks are a better metaphor for software teams. There can be a place for independence, critical thought, error seeking and cohesiveness within social networks. Social networks also capture the lumpy nature of teams and cohesion, within large teams there are clumps of high cohesion and within those clumps there are other clumps, it's fractal in nature (to some extent anyway).

Maybe the dinosaur ring is born from the statement that independance and - to some extent - error-seeking are not a natural fit for Agile projects, which thrive on close teamwork and trust. This statement is based on an assumption, that independance and, to some extent, error seeking, restrict close teamwork and trust. I think this assumption is incorrect. Team members can be part of a gelled team and yet have independance of thought and desire to find and fix defects. One of the developers I respect is extremely independant of thought, focuses in on defects with a laser accuracy, but can be seen pulling in the same direction as the other team members. If I had an agile team, there would be a place for this person, I don't want 'yes men', group polarisation and group think. Despite the pro group-think article, most teams that attempt it will fail. The effect on both the teams judgement and the host will be too detrimental.

So, I'd like to see a better metaphor, but agree entirely with the article when it talks about testing and Is there an alternate metaphor that we can build upon? One that works with trust and close teamwork, rather than independently of them? Can we minimize the need for the independent and critical judge?. I see trust, close teamwork and independance and critical thought working well in teams already, we don't need to abandon those things, just nurture them at the same time as we nurture the product.

Rationalism

2004-10-04T21:23:00.000+01:00

I like to consider myself rational, but I don't think that is always the case. I like to think I exhibit the attributes of someone born under the star sign scorpio, and not under virgo. I do things that I consider will result in some 'good', but sometimes cannot articulate what good means. Many many times in my career there have been design discussions where someone will say 'why should we do it like this' and I have only been able to say something unexpressive like 'because it is more elegant'. I'm always struggling to articulate the tacit knowledge I've built up.

This means I like to be in an environment where I'm allowed to communicate irrational things, where I don't have to follow scientific method all the time. On the other hand I've just prescribed the opposite in a meeting I've been reshaping. Before reshaping there was discussion, ideas, but nothing done. Afterwards, ideas cannot be raised in the meeting without the originator producing a brief written case for the idea ahead of time. Now we have some discussion, ideas, and things are getting done. It's not the scientific method I admire, it's the desire to have informed debate and action. In this case a rational approach worked, but it can never be the be all and end all.

Morphology of Software

2004-09-29T18:11:00.000+01:00

I read a while ago about Propp's Morphology of Fairy Tales. It made me ask myself what a Morphology of Software would look like, would there be a villain and a return, there are certainly interdictions? Seems like it would be a useful thing to have and you'd want one for different levels of granularity (enterprise through to application). Would a statement in this morphology be something like 'all ERP solutions have a business data warehouse to enable BI/MI?'.

Some people are working on the automatic generation of stories, given a morphology. Could we automatically generate a high level architecture from a morphology and some additional data? Would we want to?

Primacy of code

2004-09-29T17:32:00.000+01:00

Some extremists would say that the source should be the only place to look when I need to understand a system. No secondary sources allowed.

NLP and other approaches to understand how humans interpret the world around them point to the fact that people have different preferences for learning:

Some like to read.
Some like to have things explained to them.
Some like to view pictures.
Some like to engage in discussion.
Some like to experiment and learn through feedback.
Some like mixtures of the above.

Some need more than one go at experiencing something before they understand it.
Some need things in small chunks.

Maybe primacy of code discriminates against those who don't learn well through textual readings.

I'd like to see as a start an IDE where I can put visual comments (better still, in a way that lets it create an animated slide show) rather than just text. The IDE would place an XML description in the comment (or maybe a link to a seperate image file ) so that the compiler wouldn't barf and then render it accordingly. Ideally there would be an ALT tag and the IDE could do 'Save as Text...' and the commnets would just be the content of the ALT tag.

Ch ch ch changes...

2004-09-07T10:35:00.000+01:00

Addition/removal of attributes.

Addition/removal of entities.

Changes in cardinality of relationships. [1]

Changes of ownership in relationships. [1]

Granularity changes. [1]

Encoding changes. [1]

Unit changes. [1]

Identifier changes. [1]

You can implement the above with some set of transformations (some obvious - i.e. add attribute, some less obvious - i.e. granularity change) and to ensure backward compatibility you'd want to build a schema that, when the transformations are applied, is still understandable by actors expecting the original schema. Clearly at some point backwards compatibility has to break, either structurally or semantically.

If you are considering how to build an extensible schema the above might help in identifying what extension mechanism to use where.

[1] Ventrone, V. and Heiler, S. (1991). Semantic heterogeneity as a result of domain evolution. SIGMOD Record (ACM Special Interest Group on Management of Data), 20(4):16–20.

Software Documentation

2004-09-06T04:23:00.000+01:00

When Brian Marick had a problem with CruiseControl he rejected 'use the source Luke' because he didn't have time. This makes an interesting implicit point about software documentation; often you are too busy to use the source. If someone has provided a framework which you are going to utilise then it needs to be documented since you are always too busy. You may respond with "that's ok, that only applies to stuff like Spring, Ant, Cruise and the rest since they are tools or frameworks and designed to be treated as a black box". The 'black box' is a consequence of abstraction, and all s/w should utilise abstraction where needed to control complexity. If we follow the 'no time' argument to its end then this implies there needs to be some degree of s/w documentation for key abstractions within a s/w construction (key == (stuff we use a lot || hard stuff we use little enough we forget it)).

I think it is this that gets my goat when I'm reading code with no decent comments or class responsibilities. I know that there is no guarantee that comments are kept up-to-date, but the question in my mind is 'should we avoid using things because they not be reliable or make sure these things become reliable'? I'd prefer the latter, though its harder and if there is a solution its probably self-discipline.

SOA & Society

2004-08-25T10:01:00.000+01:00

... we are now in the days of SOA. I like to respond to this in a world weary fashion by describing how the pendulum swung from monoliths to highly distributed and back to some centre ground. On the walk in this morning (being too lazy to run in - again), it occurred to me that maybe the pendulum isn't only responding to technical forces but to social forces. For instance, we are moving back to centralised systems with thin clients and highly structured and regular architectures at the same time as Western society is becoming more authoritarian and interventionist. When CORBA came along (and the ideas that started it) it was at the tail end of an era pushing libertarian values and 'there is no such thing as society'. Take a look at Jini, it was officially launched in 1999, smack in the middle of a political change in the West that has taken us to where we are now. In many ways Jini was a technology based on loose co-operative federations of services. Highly tolerant of topology changes and removal/addition of services it harked back to less interventionist days where centralisation was an anathema. Maybe that was one reason it didn't get as much mindshare as it could have done (that and the utterly shite tooling that accompanied it). So, maybe (and that's a big maybe), for a technology to cross the chasm from early adopter and get real mindshare it must reflect the society in which it is deployed.

Equipment sizing

2004-08-24T16:31:00.000+01:00

RFP is complete, finally. So, the most boring part of its current role is complete.

Mark Twain on equipment sizing and performance:

In the space of one hundred and seventy-six years the Lower Mississippi has shortened itself two hundred and forty-two miles. This is an average of a trifle over one mile and a third per year. Therefore, any calm person, who is not blind or idiotic, can see that in the Old Oolitic Silurian Period, just a million years ago next November, the Lower Mississippi River was upward of one million three hundred thousand miles long, and stuck out over the Gulf of Mexico like a fishing-rod. And by the same token any person can see that seven hundred and forty-two years from now the Lower Mississippi will be only a mile and three-quarters long, and Cairo and New Orleans will have joined their streets together, and be plodding comfortably along under a single mayor and a mutual board of aldermen. There is something fascinating about science. One gets such wholesale returns of conjecture out of such a trifling investment of fact.
Life on the Mississippi 1883