The Future of Computing, Part 1

The OSNews is accompanied by the by-line “Exploring the Future of Computing“. In this series I’ve decided to do exactly that, to go beyond the daily stream of the latest updates and rumours and cast my eyes at the future. What will happen to Software, Hardware, the Companies and Technologies involved and how these are developed. I for one think there will be big changes to come, some for the better, some for the worse.

Predicting the future is not an easy task. If it was life would be a lot easier.
Sometimes people get it right, more often wrong. Sometimes predictions become self fulfilling prophecies, someone writes about a future invention, then someone else reads it and decides to build one…

I don’t know if what I write is going to happen but I suspect much of it will in one form or another. I’ll tell you what I think is going to happen and why, only father time can tell us if I’m right or wrong.

Part 1: The Technology industry catches up with the rest of the world

The transfer of jobs overseas in the technology industry is the continuation of a process started in manufacturing many, many years ago. Organised workers and unionisation are next, a process started well over a century ago in other industries.

One way to predict the future is to look at the past. This may sound daft in the technology industry but perhaps surprisingly it is just as true here as anywhere else. The processes that will happen in the technology industry will mirror the changes in other industries as they too have had to adapt.

Liability and Regulation become the norm

The technology industry is still young, anyone can start a company and start selling software. They don’t at this point have to worry about officially sanctioned certifications or liability, but not for long, they’re both in the post, in fact in some cases they’ve already arrived [1]. With the increased complexity of software and our ever increased dependance on it, regulation of the software industry can’t be too far away. In some countries it is illegal to call yourself an Architect or Engineer unless you have be authorised to do so, this does not really apply in the software industry at the moment but it will do.

Some fear the idea of liability because it will mean the end of programmer’s freedom to create software. No, this does not mean that, this means that for major projects you will need to be qualified to participate in the most important aspects. Programmers will become like construction workers, as long as you can churn out working code that’s fine, but the Architecture and Engineering within the system system will be done by highly qualified people and if it goes wrong they will be liable for the failure.

When errors hit critical infrastructure, things go wrong and lives get lost it’ll all begin to change. The technology industry will mature and turn into just another infrastructure industry. I for one think software would work a lot better if those creating it were liable if it failed, programmers would be a lot more careful if they knew a huge fine was waiting if they got it wrong.

In the construction industry the Architect and Engineers are liable if something goes wrong, as a result they have to be very careful in their work and need to be certified competent in their profession before they work. As a result you don’t see many buildings falling out of the skies, if you do it’s major news [2]. Despite wind rain, sun and snow – the most powerful forces in nature – there are buildings still standing after thousands of years.

The Problem with Software

In order to create the more reliable that’s going to be required the way software is developed is going to have to change. Programmers tend to use a bottom up approach, they are responsible for their own little part, there isn’t much if anything in the way of an overall design. You don’t know at the beginning what the end result will be.

Could you imagine constructing a building like this? It’d be a complete disaster! [3] [(see pages 5-6)]. Yet this is exactly how software is built, is it any wonder that so much fails. Fails? Yes, there is a problem known as “The Software Crisis” which is the fact that 88% of business software is late, works but not to specification, or just doesn’t work properly. In an industry where the practitioners pride themselves on their intelligence, how come we get it wrong nine times out of ten?

There are many reasons for this: Bad software development models is just one [4], another reason is that business people and technical people don’t understand one another, there is a miscommunication of requirements so what one wants and the other delivers are two different things. It is the attempt to fix this that causes many problems, it’s often done to software never designed to change, late in the process so there isn’t sufficient time to make the changes necessary in a good way.

There is also an unfortunate tendency in the technology industry for using young workers willing to work long hours, this has a double negative effect:
Young programmers are inexperienced, long hours mean programmers get tired and make mistakes, mistakes take many times longer to fix than make so all those extra hours go to waste. I’d bet a team of older developers will produce better code faster in 40 hour weeks than a bunch of recent graduates working 60 (or more) hours.

An Opportunity for Open Source : No

Some may consider that Open Source is the answer to this, but that’s because Open Source gets it’s reputation from a relatively small number of successful projects all of which have a large number of contributors and active members. For the most part these are done as a hobby, for the sheer love of it. Unfortunately software produced by companies is done for rather different reasons and love isn’t one of them. Opening your source code will not automatically guarantee your software will get developed at all never mind on time. It does however guarantee potential attackers can see if there are holes in your software.

Programmers in business are under tight schedules to deliver so there is no time for love, time consuming optimal solutions are forgone in favour of quicker implemented approaches. Open Source teams may have found a solution to these problems but without the love this technique won’t help business.

The Real Problem

The biggest problem is that software is complex and as it grows it becomes exponentially more complex, no single developer is going to see the whole picture. Even if they could the skills required to understand the big picture are different from those required at the code level. Coding requires a good memory of techniques and the ability to use the right techniques to solve the right problems, creativity isn’t really a requirement.

The big picture, the Architecture, is a different area altogether, it requires the ability to solve problems without an answer, it requires creativity, the ability to design. Software developers are trying to solve this problem by applying the techniques of software development. Software development can be taught: you have this problem, you use this solution. Design is a skill, as such is also great deal more difficult to teach it.

Software Architecture will become popular, then a legal requirement

The original idea behind Software Architecture is to use a top down approach – to produce a design first then start developing. Unfortunately this method (referred to as the waterfall method) doesn’t work very well for systems which change and large systems tend to change.

One method which does work is a formalised method of the bottom up approach called Extreme Programming [5]. This method advocates the use of small iterative development stages and constant refactoring (the cleaning up of software). In Extreme programming there is no large design phase at the start, instead the refactoring technique allows a design to evolve into existence. You don’t know at the beginning what the end result will be. While this may sound counter intuitive for non-programmers this method works very well. The changes which mess up so many systems are caught and handled by the refactoring.

Modern Software Architecture methods have a large initial design phase but the design is continued alongside an iterative development process. Actual code tends to move away from the initial design so the design is tracked as it changes, when a new requirement is added or something needs to be changed the Architect can look at the overall design and modify the design before coding of that change begins. This is process is similar to the refactoring used in Extreme Programming but is done at the Architectural level first and later at the code level

The benefit of this approach is that someone always knows what they are actually building, what fits in where and effects what, as such it is much easier to make changes and keep the software maintained. Some Architects already use a iterative Architectural technique and Extreme Programming as the development model with success.

Managers and developers may currently see Architecture as an extraneous phase but when businesses realise the productivity gains and cost savings it can deliver it will come into focus.
If a change at the design phase takes 1 day, it will take 10 days if made at the development stage and 100 days if made in the maintenance phase. Given software spends most of it’s life in the maintenance phase Software Architecture makes a lot of commercial sense.

When liability becomes a major issue the reliability and accountability Architecture can deliver will be seen as the key to producing better software. Software Architecture will becomes a “real” job complete with real official certifications and liability. I don’t mean company based certifications as is common in the technology industry today, these will be industry wide with legal standing. It’ll be a difficult, risky job but we can learn from the past, the exact same thing happened in the building industry many years ago.

Some people have seen this process coming and formed their own organisation [6]. There are many books to be read for the aspiring Software Architect. Many programmers already have the requisite skills but probably don’t know it, Architecture is a different area and you’ll find out if it’s the area for you by reading some of these books which cover the different aspects [7] [8].

Unlike the building industry software development can be closely monitored, I expect we will see real time tools for visualising and monitoring architecture. I also expect that development models will evolve. Some models are huge and require a great deal of complex and expensive tools, others are too simple and end up with chaos. Consequently I expect we will eventually find a happy medium where an extensive design phase is followed by a process similar to Extreme Programming. I also expect testing will also play a much greater role in development.

Wither the Penguin?

Some have suggested that Liability will kill off open source, is this true? Will the penguins shuffle off back to the South pole?
I do not expect this will be a problem for Open Source development, it could in fact have quite the opposite effect and make it more popular than ever. If you are liable for your product you will be very keen on using tried and trusted technology. I expect the model used by Java and Perl programmers will become more popular, they use a great deal of Open Source modules in their systems.

For Operating Systems IBM, HP and friends can all control and make sure their own in House Unix systems are ready for liability, Unix isn’t dead yet by any means and this may keep it alive for a long time yet. I don’t expect the BSDs will have too many problems, they are generally created by smaller teams and FreeBSD, OpenBSD NetBSD all focus on stability anyway. OpenBSD focuses on security but stability is a pre-requisite for this. NetBSD is more research orientated but it’s ultra-portability means it too has to be a highly stable system.

Liability may prove difficult however for other systems:

Small modules even of unknown origin can be checked but it’s a different story for large monolithic systems such as linux, would you trust your life to Linux? Given it’s distributed development model who’s going to accept liability for it? This wont kill Linux but will lock it out of some applications until these issues are solved.

Microsoft will have to accept liability for Windows, when they decide to do another security binge they will have to do more than a marketing excercise. Microsoft have built their empire by building software which works most of the time, I expect they will have a hard time adapting to developing software which works all of the time.

Software Patents and Other Issues

Patents will not cause the death of the industry some predict, this is typical computer industry over-hyping. Patents are a double edged sword for the computer industry. They do allow big companies to stifle competition but on the other hand they also allow chancers (a person who takes a chance) to sue the very same big companies for very large sums of money – just ask Microsoft. I expect some companies will become rather less enthusiastic about patents when they’ve had to pay out a few billion dollars.
I think this may lead to a change in the patents are issued and administered, I also expect (or rather, hope) that patents will eventually be issued for shorter lengths of time as befits a rapidly developing industry, this will benefit everyone from the big companies to open source developers, it will also spur innovation since you wont be able to live off past glories for long.

DMCA and such like laws will be seen for exactly they are: a complete overreaction.
They will eventually be toned down to sensible levels. This may happen in court room precedents though rather than in government. In Europe it’ll happen the same way it always has with stupid laws: we’ll just ignore them. Some countries will probably not implement them in the way intended and thus take out their bite.

I’d like to see government mandated data exchange formats, i.e. standard office use file formats.
This would do more against Microsoft’s desktop monopoly in one night than Linux has done in ten years. Unfortunately I don’t see any move towards this whatsoever.

I fully expect SCO to lose their case and go bankrupt.
The management will not care because they got rich anyway.
This will raise questions and investors will end up suing SCO, this could get messy but could potentially lead to new company regulations in the US.
Germany already appears to have these regulations so they wont care either.

Banging the DRM

DRM is not going to work. In some cases such as Digital TV, it’s possible to control both the source and destination for the media yet even these sophisticated DRM systems are cracked. The only reason the TV companies can keep making money is because they still control the SetTopBox and can rapidly update the decryption keys.

DRM for mass market media is a different matter altogether, the companies do not control the media players and thus cannot update them. Once one of these DRM systems is broken that’s it – it’s broken forever. Once someone implements a working anonymous file sharing network it’ll all get shared.

Old style copy protection mechanisms worked because even when they are broken there was no way to get copies to everyone in an easy way. These days we have the internet so it doesn’t matter how difficult the DRM system is to break, it only needs to be broken once and that’s it, everyone can copy the files.

The future is flat rate media [9], pay $10 a month and listen to whatever you want whenever you want, royalties are distributed according to what’s being listened to. There will no need for DRM as there’s no point trying to cheat, you could try and download everything and stop paying but it’ll be more expensive to store it than pay the $10, more importantly it’ll be easier to pay the $10 so even if a few do cheat 99% of people wont be bothered.

That’s part one, next time I’ll cover the radical changes I expect to happen in the hardware domain, some of which will leave the industry reeling. I also cover how Microsoft will attempt to reverse the trends and try to re-gain it’s monopoly in a way nobody will expect.

Part 1 References

[1] Some organisations already legally require an Architecture

Enterprise Architecture by Legislation

[2] When a building falls it’s news:

Many die as Turkey flats collapse

http://news.bbc.co.uk/2/hi/europe/3453131.stm

[3] The Software Architect’s Profession – Mark Sewell and Laura Sewell.

A philosophical look at Software Architecture and why we need it (scroll down page):

http://www.wwisa.org/wwisamain/books.htm

[4] Article on software development myths

The Demise of the Waterfall Model Is Imminent” and Other Urban Myths

[5] Extreme Programming

http://www.extremeprogramming.org/

[6] World Wide Institute of Software Architects

http://www.wwisa.org

[7] Software Architect – By Nigel Leeming

What do they do? Read this on-line book to find out:

http://www.softwarearchitect.biz/arch.htm

http://www.softwarearchitect.biz/frames.htm (Older browsers)

[8] Other books on modern Software Architecture:

Software Architect Bootcamp – By Raphael C. Malveau,Thomas J. Mowbray.

Want to be an Architect? Get your hair cut short and enlist for basic training here (scroll down page):

http://www.wwisa.org/wwisamain/books.htm

Software Architecture: Organizational Principles and Patterns – By David M. Dikel, David Kane and James R. Wilson.

http://www.vraps.com/index.jsp

[9] I’m not the only one who thinks flat rate media is the future:

http://www.theregister.co.uk/content/6/35260.html

Copyright (c) Nicholas Blachford, February-March 2004

Disclaimer:
This series is about the future and as such is nothing more than informed speculation on my part. I suggest future possibilities and actions which companies may take but this does not mean that they will take them or are even considering them.

59 Comments

  1. 2004-02-09 6:21 am
  2. 2004-02-09 6:41 am
  3. 2004-02-09 6:57 am
  4. 2004-02-09 6:57 am
  5. 2004-02-09 7:32 am
  6. 2004-02-09 7:56 am
  7. 2004-02-09 7:57 am
  8. 2004-02-09 8:10 am
  9. 2004-02-09 8:26 am
  10. 2004-02-09 8:50 am
  11. 2004-02-09 9:15 am
  12. 2004-02-09 9:42 am
  13. 2004-02-09 9:45 am
  14. 2004-02-09 10:20 am
  15. 2004-02-09 10:35 am
  16. 2004-02-09 11:41 am
  17. 2004-02-09 12:00 pm
  18. 2004-02-09 1:51 pm
  19. 2004-02-09 2:05 pm
  20. 2004-02-09 2:08 pm
  21. 2004-02-09 2:09 pm
  22. 2004-02-09 2:17 pm
  23. 2004-02-09 2:28 pm
  24. 2004-02-09 2:29 pm
  25. 2004-02-09 2:48 pm
  26. 2004-02-09 2:53 pm
  27. 2004-02-09 2:58 pm
  28. 2004-02-09 3:13 pm
  29. 2004-02-09 3:30 pm
  30. 2004-02-09 3:38 pm
  31. 2004-02-09 3:51 pm
  32. 2004-02-09 4:11 pm
  33. 2004-02-09 4:25 pm
  34. 2004-02-09 4:33 pm
  35. 2004-02-09 4:52 pm
  36. 2004-02-09 5:09 pm
  37. 2004-02-09 5:13 pm
  38. 2004-02-09 5:19 pm
  39. 2004-02-09 6:06 pm
  40. 2004-02-09 6:17 pm
  41. 2004-02-09 6:23 pm
  42. 2004-02-09 7:01 pm
  43. 2004-02-09 7:08 pm
  44. 2004-02-09 7:13 pm
  45. 2004-02-09 7:23 pm
  46. 2004-02-09 7:27 pm
  47. 2004-02-09 7:29 pm
  48. 2004-02-09 9:11 pm
  49. 2004-02-09 11:06 pm
  50. 2004-02-10 3:08 am
  51. 2004-02-10 4:45 am
  52. 2004-02-10 4:47 am
  53. 2004-02-10 2:14 pm
  54. 2004-02-10 4:47 pm
  55. 2004-02-10 11:54 pm
  56. 2004-02-11 1:21 am
  57. 2004-02-11 3:34 am
  58. 2004-02-11 10:09 am
  59. 2004-02-11 10:26 am