The Internet Services Providers Association

Response to the Smith Group Report for the Home Office on Technical and Cost Issues Associated with Interception of the Internet

23 May 2000

Prepared by ISPA Council

Executive Summary


Just before Easter, the Home Office published a report they had commissioned on the cost of warranted interception at Internet Service Providers (ISPs). The report was written by consultants from The Smith Group Ltd and can be found at:

There have been a few previous attempts to quantify the cost of interception to the ISP industry, which have all come to the conclusion that it was going to be expensive. In the event, the consultant's analysis has taken a number of factors into account that had previously been overlooked and to that extent contributes something to the debate.

Filtering and Mediation

The Smith Report proposes three schemes for capturing data. In each case the data is fed into a filtering and mediation system - which cleans it up and hands it over to GTAC (to be sited in Thames House, London) in a secure manner.

This filter/mediation system would be pretty much the same at every ISP and the Report recommends that Government should fund this on a central, once off, basis to produce a standard system.

There is the hint that similar software already exists, presumably developed for GCHQ in Cheltenham. The report concludes that it can be adapted for ISP interception purposes and rolled out for just over 500,000.

Although simple enough in theory, this software is likely to be complex in practice, especially where it is attempting to reconstruct real-time data flows that may choose different routes for different packets. Without knowing how far existing development of such software has come, it is hard to comment upon the accuracy of the Report's estimates of what it will cost to develop for ISP networks. But in any case, several decades of experience with cost overruns in Government computing projects should make one expect that a great deal more will be spent in the end than was ever originally envisaged.

With this substantial expense in mind, ISPA naturally welcomes the Report's recommendation that this software development should be entirely funded by the Government - although this is not current Government policy. Perhaps the Smith Group will have more influence on Home Office views than industry has?

Active Interception

The first of the three schemes that is proposed by the Smith Report is the capture of email. This is called "active interception" and the Report recommends that this should be universally applied.

However, Charles Clark, the Minister of State at the Home Office has indicated otherwise! In the debate on the Report Stage of the Regulation of Investigatory Powers (RIP) Bill he said, "We do not expect to ask all ISPs to carry an intercept capability".

Universal or not, active interception is clearly thought by the Reports author's to be a cheap option... but is still has an estimated cost, for even the smallest ISP, of 44,700 in the first year (19,400 each year thereafter). The Report suggests that by splitting the costs in line with current Government policy the ISP will have to pay a mere 18,800 of this (9,400 in subsequent years). This expense will be sufficient to drive a large majority of UK ISPs out of business. The elimination of the smaller ISPs will eliminate innovation and competition.

The costs for large ISPs are somewhat larger. The Report suggests 113,300 in the first year with the ISP bearing 41,900 of this - though ISPA's larger members suggest to us that the complexity of their email systems have not been understood and delivering wholesale email interception for this sort of price is extremely ambitious.

This is an industry where few companies are making any profits but are instead investing for the longer term. Therefore, these sort of costs - modest though they may seem to a Government investing a reported 25,000,000 in their end of the interception pipeline - are likely to make the difference for some of ISPA's members between business success and failure. The anticipation of incurring such costs is likely to deter new entrants to the industry, thereby reducing innovation and competition. The Internet is a global marketplace and these costs, only applied in the UK, will act to disadvantage British companies. An equivalent scheme (CALEA) in the USA has run entirely on Government funding.

We suspect that the Home Office led the Smith Group towards proposing a universal scheme for capturing email not because of operational need - but because it was seen to be "fair" to require all ISPs to provide the same functionality. ISPA views this universal requirement as being equally unfair - and insists, once again, that the only sensible way forward is for all funding to come from Law Enforcement Agency budgets. This would allow a proper prioritisation to be made between rolling out a capability at ISPs and, for example, putting more crime cars into rural areas.

However, if Charles Clark is indeed overruling the Report's recommendation that all ISPs should fit active interception, then there will not be a "level playing field". If the Government can impose costs on ISPs pretty much at random then this will be inherently unfair to those who are burdened in this manner. The only fair way of proceeding in a world where interception is patchily applied will be for the whole cost to be borne by the Government.

To summarise: If there is an operational need to intercept email at an ISP, large or small, then of course it should be done. However, to have the ISP pay for it and risk putting that ISP out of business is nonsensical. It is equally nonsensical to force the entire industry to adopt an unproven technology, one that industry experts know will not work. The Internet's stunning success is a direct result of just the opposite approach: the Internet adopts new technology only incrementally, only after it has been proven to work.

Passive and Semi-Active Interception

The problem with email interception at ISP servers is that email may be held elsewhere (on American web systems such as HotMail perhaps). Also, other sorts of communication may be used (chat rooms, voice over IP, NetMeeting, ICQ etc, etc, etc). In order to intercept with this sort of rapidly growing generality it is necessary to capture raw Internet Protocol (IP) streams.

Interception at this level is a series of challenging tasks. The data has been split into individual packets, which may be travelling by different routes, and so the first step is to reconstruct the data stream. It will then be necessary to interpret this stream by deducing which of many software programs generated the traffic. Finally, the reconstructed voice or text must be analysed and useful intelligence generated from it.

The Smith Report proposes two schemes for raw data interception. The passive method requires that all traffic is monitored and then the data coming from an individual of interest is pulled out. The semi-active method requires the ISP network to be dynamically reconfigured so that the traffic that is of interest is specially routed to pass by an interception point. This dynamic scheme is far more complex to deploy, but the data rates are less and so the filtering is cheaper.

From a technical point of view ISPA has serious doubts that the consultants have understood the speeds and bandwidths of the links that are currently deployed by the largest ISPs, let alone the sort of speeds that will soon be in use.

The consultants also seemed to believe that ISPs run with a "backbone" where all traffic can be readily seized. They draw this backbone picture several times in the report. This is simply wrong as a generic description of ISP networks - network designers go out of their way to avoid "backbones" because they lead to single points of failure - and at even the smallest of operations there will be a mesh of connections all of which will require to be intercepted. So the costs, particularly of the passive scheme, will be far higher than the consultants have calculated. We know of no one with substantial experience of the Internet who believes that this type of interception is possible at any reasonable cost. Unless the Internet is to be slowed to a crawl, it would require equipment which is not available to anyone in the world.

Nevertheless, even the consultant's sums show that the first year cost for the passive scheme at a large ISP will be a massive 1,384,000. Well under half of this sum will be paid for by the Government, with the rest being, effectively, a special interception tax upon the ISP industry.

The consultants have seen even this underestimate as somewhat expensive and have therefore recommended their bargain basement "semi-active" scheme which they calculate to cost a mere 217,300.

Unfortunately, they've got their sums on this completely wrong.

They have assumed that by specially routing the traffic, they can make do with a single interception point for the whole of an ISP's operations. However, with the largest ISPs operating out of several facilities centres from Docklands to the Clyde it is pretty clear that arranging for traffic to pass a single point is likely to make the existence of an interception obvious to the technically savvy. A number of medium sized ISPs will have similar problems.

The semi-active scheme may, ISPA accepts, turn out to be cheaper than passive - but it will need far more than one interception point at the large ISPs and so it is likely to save only a few hundred thousand pounds rather than a million.

In summary: ISPA's view of raw IP stream interception is that it is a complex task and will be impossible to make work successfully. It will, without doubt, be extremely expensive. The Smith Report does have what on superficial inspection looks like a viable technical approach, but in their ignorance of how large ISPs have built their networks they have ignored a number of practical issues. A realistic view is that the cost per large ISP must be seen to be at least a million pounds. ISPA does not believe that the Government should be taxing the ISPs (at a rate of about 10% of their network costs) in this way.

Once again, the key point needs to be made - if interception of raw IP streams is a cost-effective thing to do, then law enforcement budgets should be paying for it.

Technical flaws

The naive assumption that ISPs have "backbone" networks has already been referred to, but there are a number of other dubious assumptions within the Report.

The Smith Report has been based on an underlying model of intercepting about one in every 10,000 data streams. They have sized their systems by assuming that ISP data rates can be divided down by a factor of 10,000. However, until they have applied their filtering systems, the full rate data streams will have to be intercepted and the small number of required packets pulled out.

ISPA notes the list of equipment in section 4.6.4, but is not convinced that devices to intercept and filter packets are commercially available at sensible prices. Even the smallest ISP will be using 100Mb Ethernet and large operations will be using gigabit Ethernet or ATM at STM4 speeds or faster. i.e.: it is unclear that the basic interception design will work except at the edges of the network where speeds are lower. Interception at the edges is more expensive than centralising the capability because there are more points to tap.

The "semi-active" interception scheme seems to have been devised by someone who has believed the blandishments of salesmen. In the real world, there are not spare ports on switches, applying filtering rules causes a heavy performance penalty on the network or starts to "tickle" bugs in firmware implementations. It is also far from clear that policy-based routing set up in a rapidly changing dynamic manner to deal with interception targets making frequent calls will not cause a significant loss of performance for the ISP's other customers. The UK's ISPs have unmatched expertise in this area. We don't know how to do what the Smith report proposes be done. For the government to accept these simplistic recommendations would be simply foolish.

One of ISPA's member has conducted some rudimentary tests on two of the popular high performance router types used in ISP networks, the specific configuration of process switched routing showed a decrease in performance of approximately 90% and 70% respectively. We would be interested to see the results of any testing the Home Office has conducted in this regard! Accordingly we believe that the Smith report as a blueprint for interception exhibits the classic triumph of hope over experience that leads so many computing project estimates to be miles wide of the mark.

Additionally ISPs fear that the mechanisms being proposed were not designed for the regular reconfiguration required to trap specific intercepts, and point out that no allowance in costs has been made for the huge expense of a network outage caused by faulty routing.

No consideration has been given to the problems of virtual ISPs. There is a range of business models ranging from badging of services up to maintenance of complete sets of infrastructure. The BT IP Dial system raises similar problems. Authentication is performed by the RADIUS server at the ISP who owns the customer, but subsequent traffic routing is done by BT. The customer may never access the ISPs systems during their online session. Without serving warrants on all of the organisations involved in providing the customer with connectivity, it is unclear that interception will be possible.

Active interception requires modification to server software. This will limit ISPs to using server software that is either "modifiable" (i.e. open source products) or that has the UK government modifications applied. This will limit the range of products available to the ISP to use. Similar considerations apply to authentication systems such as RADIUS or RADIUS replacements.

A common trend is towards complete off-the-shelf systems (often called "appliances"). Since these come from other regulatory environments they will not have interception capabilities built-in, or at least not for the standard price. Preventing their deployment because they cannot provide email interception capabilities is likely to be damaging to competition and product innovation. In other words, imposing these policies would rapidly turn the UK into a technical backwater.

Categorising ISPs

The categorisation of ISPs into two groups, small and large, is overly simplistic, even if it has only been done to illustrate the wide range of expense of interception systems.

Large ISPs can often be indistinguishable from major Telcos and even the smaller ISPs tend to have highly varied businesses. This is a reflection of the different backgrounds that ISPs come from, and also a result of the high level of competition and rapid rate of innovation that exists in the industry.

If some requirements are only to be applied to particular sizes of ISP then there will be serious problems in trying to develop suitable categories. Examination of turnover, profit or number of customers is likely to be seriously misleading.

It will even be hard to determine what is or is not an ISP. There exist businesses that sell Internet connectivity as an add-on to their main usage of a leased line. Educational establishments may be viewed as an ISP when one considers how their students access the Internet. Small web hosting companies may offer dialup connectivity as an extra to their main business, as indeed might a Hotel or a School, where the public are allowed to use the systems for a fee.

The Government seems to be keen to encourage universal access to the Internet. It is unclear to ISPA why innovative schemes, perhaps providing Internet Cafes in pubs or supermarkets, should have to put into their business plans the risk that the Home Office will suddenly appear and require them to pay for interception capabilities.

Furthermore, the report does not take into consideration the issues of ISPs operating across national boundaries. It is not unusual for even the smallest ISPs to have servers hosted outside the UK. Similarly it is not unusual for UK ISPs to be procuring virtual Internet presences from overseas organisations or offering guest privileges on their own networks to customers who are authenticated against credentials held on the other side of the planet. There are likely to be significant legal, practical and procedural problems.


ISPA welcomes the Smith Report in the spirit in which it was provided, to inform us all of the likely cost of interception. However, it is only a start of that process. In many places the Report is naive and superficial - which is perhaps more a reflection of the complexity of the problem and the time given rather than the authors' abilities.

ISPA certainly does not welcome the Report's conclusions - that interception will be fearfully expensive - and warns that if even a small part of this cost has to be borne by the industry then it will lead to business failures, the curtailing of plans for growth, lack of competitiveness with overseas operations and less innovation throughout the marketplace.

Even if the entire cost comes from taxpayer, ISPs would far rather be using scarce talents and manpower to grow their businesses rather than creating systems that will provoke fears about privacy in the innocent and seem entirely likely to reveal that the bad guys are using unbreakable encryption.

The Smith Report was fatally flawed by its terms of reference:

The authors have not considered the more general picture, viz.: how much material (especially that which refers to illegal activity) will actually be useable unencrypted clear text.

There is a general trend towards using secure transports (such as SSH) for private material, such as email read via web interfaces. There is also much more of an emphasis on end-user tools for encryption. Customers have asked for years whether email is private. Easy to use systems for tools such as PGP allow ISPs to avoid ducking this question by waxing lyrical about their probity but will instead allow users to provide their own security.

Much of the approach of the report is to look for uniform provision rather than asking where resources should be deployed so as to get value for money.

There is no consideration of the practical and legal burdens placed on ISPs. Having a facility built in to their networks that allows warranted interception means that great care will have to be taken to ensure that it is not used for any other purpose.

This will add unwanted complexity to many operational issues. If there are security breaches it will lead to further costs to ISPs who will have to compensate customers for their loss of privacy through the use of equipment that was not operationally necessary, but was only fitted to assist the Government.

The report completely ignores the costs of network failure/performance hit by using the semi-active scheme of interception.


The Government has talked a lot about its consultations with industry - which comes as somewhat some surprise to ISPA. In several months of discussions we have hardly clocked up a full six hours of talks with the Home Office and so we're still at the stage of explaining the problems and have hardly started to look for practical solutions.

We still have no idea what the Order made under Section 12 will look like, nor how a notice giving effect to that order on a particular ISP is likely to be expressed.

The Smith Report is a view under the Home Office's veil. But we remain extremely unhappy about the impending nuptials - not least because it seems that we will be asked to pay for a devastating cost for technology that will not work.