13 February 2016

Train Control Update

There have been two recent and important developments in the area of train control systems, the safety systems known in U.S. parlance as "Positive Train Control" or PTC.  Both of them have a direct impact on the future of the peninsula rail corridor.

CBOSS Goes Sideways

Caltrain's CBOSS project, criticized for years on this blog, is in ever deeper trouble.  According to Caltrain's latest project update, the $231 million previously allocated for this project are nearly spent, but the project is way behind schedule and struggling in the most perilous and delay-prone phase of all: testing of the integrated system.

Not Ready for FRA
Testing is where it all finally comes together, not in the carefully controlled environment of a lab test bench, but in the real world with all its ugly imperfections, annoying glitches, and external influences.  In each segment of the railroad, three things have to happen in sequence: (1) everything has to be fiddled with until it works; (2) when everything works, a dry-run of the official acceptance test, known as "Pilot Testing," is performed; and (3) the system is formally accepted after passing the official FRA-witnessed test of all its functions.  Caltrain's project is stuck in the "fiddle with it until it works" phase, helpfully diagrammed as an infinite loop in the test flow diagram at right, extracted from a CBOSS Verification, Testing and Inspection Plan filed with the FRA.  This is where the schedule and budget are blowing up, with Caltrain's next step described as "Complete Segment #3 Pilot Testing and FRA Witness Testing," coming as soon as they are "Ready for FRA," whenever that may actually be.

Caltrain's update mentions "software release delays" relating to the I-ITCS product that CBOSS is based on.  There are worrying signs that I-ITCS may become a technological dead end: the company that makes it, GE Signalling, was recently acquired by French rail giant Alstom.  Alstom's mainline signaling product portfolio does not give it top billing.  Furthermore, the FRA process is described as being "in flux", meaning that the goal posts are moving.

What we have here is a classic foundering IT project, and it isn't clear if throwing more money at it (at a burn rate of about $50M/year) is going to save it.  With the federal PTC implementation deadline now pushed out to 2018, this is a good time to stop and re-assess the project before escalating the commitment.

HSR Buys Radio Spectrum

Meanwhile, the California HSR Authority is about to spend $50 million to secure the rights to a key chunk of radio frequency spectrum.  The frequency bands being purchased are 757-758 MHz and 787-788 MHz, not the usual 220 MHz band used for freight PTC systems.  Instead, the CHSRA has documented its intent to deploy ERTMS, an increasingly mature and proven train control standard that originated in Europe and is increasingly in worldwide use.  The two bands purchased for HSR are not sufficiently wide to deploy GSM-R, the obsolescent communications standard currently used as part of ERTMS.  It is more likely that California's deployment of ERTMS will use a more modern, secure and spectrum-efficient LTE communications layer, following the evolutionary path beyond GSM-R already being planned for ERTMS.

Connecting the Dots

Suppose the following conditions come to pass:
  1. CBOSS proves unworkable (increasingly likely)
  2. HSR shifts its focus to Northern California (possible)
  3. HSR finalizes plans for ERTMS as its high-speed train control standard (very likely)
  4. Due to construction delays, HSR needs new and productive ways to spend federal funds that expire by 2017 (possible)
Then an opportunity exists to deploy a train control pilot project on the peninsula rail corridor, using ERTMS with LTE communications in the 700 MHz band.  This scenario recognizes an important fact so far disregarded by Caltrain, that HSR will become by far the largest tenant railroad on the peninsula.  Ignoring this fact is an odd position to take for a railroad that hangs its future on "blending" with HSR.  Caltrain will surely dislike the idea, bleating about closer headways, crossing signal integration, station stop enforcement and other completely unproven bells and whistles--as they have since 2009--but events are now quite clearly bearing out the relative technological merits of ERTMS and CBOSS.  It's just sad that it took a quarter of a billion dollars to settle the question.

In the unforgiving world of system integration testing, reality always wins.

27 comments:

  1. Clem, your articles are always so helpful. Thanks for doing what you do!

    ReplyDelete
  2. Caltrain's pattern is to buy whatever its favored consultants are selling, no matter how ludicrous or self-destructive. Doing the right thing--what Clem suggests--would take leadership of a caliber not seen in California since the days of Pat Brown.

    ReplyDelete
  3. Well, ERTMS is not currently certified by FRA to meet the PTC mandate on mix-operation. (If) any replacement of I-ITCS as the underlying system to CBOSS will still need to meet the inter-operability requirement and thus need to talk the same "language" as I-ETMS, I-ITCS, etc... And any non-standard features that Caltrain wants, would need to be included in the replacement as well, so there is no guarantee the testing and certification would be anymore trouble-free comparing to CBOSS.

    ReplyDelete
    Replies
    1. That's really the crux of the problem. Pile on that the fact that UP wants Caltrain to pay for interoperability, things become really skewed towards CBOSS/PTC.

      One thing that would help us put things in perspective:
      1) What customizations on top of PTC did Caltrain need or want for CBOSS.
      2) Are customizations in #1 required or desirable to make things smoother. For example (eliminate double gate down times when train approaches a platform before a crossing.
      3) Which of these in #1 are supported by ERTMS.

      Delete
    2. What I object to is Caltrain's organizational belief that they are a special snowflake that requires special solutions. They end up trying to invent a better mousetrap and incur all the cost and risk of doing so, at your and my expense... not just in taxpayer dollars but also in opportunity cost of worthwhile improvements not made.

      Train control is not one of those projects that you do your own way, and that hard truth is not yet understood at Caltrain. Organizations far larger than Caltrain have spent massive sums on developing technology standards, but Caltrain apparently knows better and needs to pile on a whole additional layer of customization. That special snowflake approach needs to stop, and salvaging CBOSS (if there is anything salvageable) will start with cutting loose those bells and whistles.

      The issue of interoperability need not be the root of boundless complexity--it is being used as an excuse to over complicate everything. UPRR will run I-ETMS as will ACE and Amtrak. Keep I-ETMS on 2 tracks from CP Coast to San Jose, and no more. Don't mingle.

      For the peninsula rail corridor, use ERTMS level 2, exactly the same stuff used for HSR.

      For UPRR freights coming up the peninsula, use time separation (operate from 1 to 5 AM) and treat the system as dark territory. The risk profile (in terms of traffic, hazmat loads, etc.) during that window does not warrant the use of PTC in the first place.

      And if someone doesn't like that, then it's better to buy out the freight customers and wind it all down (big bad UPRR couldn't care less!). That's expensive, sure, but it pales in comparison to the sums pissed away on CBOSS.

      And forget about the Coast Daylight. It doesn't belong anywhere except the 1950s.

      Segregation (temporally or per track) is the only reasonable solution.

      Delete
    3. William,

      The good news (and the bad news) is that the solution that Caltrain is now using was considered sufficiently one-off by the FRA that Caltrain had to get approval of the system as well as their project so the path to getting that done has already been traveled...

      Delete
    4. William writes:

      Well, ERTMS is not currently certified by FRA to meet the PTC mandate on mix-operation. (If) any replacement of I-ITCS as the underlying system to CBOSS will still need to meet the inter-operability requirement and thus need to talk the same "language" as I-ETMS, I-ITCS, etc.. Wabtec WIus for ETMs, GE ITCS RBCs for ITCS.

      To the very very, vest of my knowledge, ITCS does not in fact speak "the same language" as ETMS. If you look at what GE Transportation Systems is saying, part of the CBSOSS development is to *make* ITCS interoperable.

      Llast I looked, that interoperabilty was *not* "talking the same language", but rather a single back-end, with duplicated WIUs for ETMS and ITCS. One each for English and French, as it were. It's hard to find technical details, but here's the most recent I can find:

      http://www.apta.com/mc/rail/previous/2012/presentations/Presentations/Simonelli-L-High-Speed-Rail-Integration.pdf

      See p. 9. The UP locos speak ETMs, the Caltrain locos speak ITCS.
      But note that this shows Caltrain CBOSS, aka vaporware.

      Delete
    5. Clem,

      I recall GSM-R uses 4Mhz frequency bandwdidth, per direction (uplink,downlink) Te bands you say CHSRA bought are half that. Are they planning to run over GPRS? That's one way to reduce RF spectrum (or equivalently, increase capacity of given spectrum).

      Hm. I see India runs GSM-R with 1.6Mz uplink/downlink. Clem, do you know how that operates? fewer 200Khz GMSK channels?

      Delete
    6. My guess as stated in the post is LTE, not GPRS.

      Delete
    7. Oh, so you did. My apologies; I'm not well and somehow I missed that.

      I''m skeptical that CHSRA's contractor is thinking that far ahead, but you may be right. Then again, the last published information I saw for the ICS had CP-5 as a contract for track: no overhead catenary, no substations, no signalling.

      Given the glacial pace of standardizing GPRS as an alternative to GSM-R, I think you're courageously optimistic. Or maybe very, very pessimistic about a deployment date.

      Delete
    8. The challenge in Europe is that there is a significant installed base of GSM-R networks, making the transition to GPRS or even 4G (LTE-R) logistically difficult and cost-ineffective. In a greenfield installation like California, with system-wide RF spectrum ready and available, it's a lot easier. I don't know how mature the LTE-R standards are just yet, but the timing may dovetail quite nicely with California HSR.

      Delete
    9. As far as I know, ETCS over LTE is /.. nonexistent, unless you're a telco vendor like Nokia trying to sell "overlays" with LTe for voice and (passenger0 data, overlaied on GSM-R.

      Clem, ETCS over GPRS has seen "coming soon" for 4 or 5 years, and it's still not here. I am (or was) in the TCP/Ip research community. The phrase "Bell-shaped" head" was well-known; people would even self-apply it. (Usually people from Bell Labs/Lucent, who realized they'd said something dumb.)

      GSM-R is *virtual circuits*. 2G mobile telephony, a 20 year old design .
      You can't (re0activate even a GPRS phone in first-world countries anymore. (I tried reactivating an EVDO-generation phone when my previous phone broke; the telcos won't do it.) In another 10 years, 2G/GSM-R wlll be end-of-availability. If you can't buy the hardware, you can't keep using it, because you can't add new trains. *That* will force migration, if nothing else does. And yes, incremental deployment .. isn't really incremental, it's lineside-on-a-whole-route first, trainset deployment in parallel, and you only get to use it when both train and lineside have it.

      That said, I keep reading that the UK will "go it alone" on ETCS over GPRS, because they need it for ECML (and CrossLink) traffic density, and they don't have spare RF frequency.

      Delete
  4. CBOSS not working? Behind shedule? Cost overruns?

    My gosh, who *ever* could have predicted *that*??

    ReplyDelete
  5. UPRR could care less about the peninsula freight rights. But my guess is that they care massively about precedent and valuation comps. They are likely sitting on contiguous transit corridors in dozens of urban areas across the country. Once a value is established, eminent domain on many of these corridors becomes more likely for any government agency that wants to build infrastructure. So they won't let the rights go cheap, if at all.

    ReplyDelete
  6. @Strabo, how much less could UPRR care? ... or, maybe -- and it sounds like you did -- you meant "could NOT care less"?

    ReplyDelete
  7. Clem, You and others have said that you saw this coming. I have suggested on numerous occasions that you or Richard M. bring this to the attention of NBC 11 Investigative Unit (or any other station or watchdog agency/group), yet nobody has done so. At first I thought you and Richard may have been blowing smoke about CBOSS and other Caltrain projects but what you guys have been saying has been panning out. And now Caltrain say it will cost $42 million to put wi-fi on the trains! Why the inaction on your parts? Do you not know that these TV news stations exist? This is our taxpayer and riders money that Caltrain is squandering. Money that could be better spent running better train service, as you guys have pointed out a number of times. What about Roland, maybe you can bring this to the attention of the TV news?

    It does help, NBC 11 revealed that something like 70% of BART security cameras were fake and now BART is doing something about it, fake cameras are being replaced with real cameras.

    You guys have the technical expertise to discuss these issues with the news, I don't.

    ReplyDelete
    Replies
    1. I don't believe this would help Caltrain achieve the goal of a functioning, efficient, compatible and affordable train control system. The only relevant questions relate to how this goal can be achieved. Everything else is just noise.

      Delete
    2. Clem: "I don't believe this would help Caltrain achieve the goal..."

      Whatever!

      What makes you think that posting here on these obscure little blogs will help Caltrain achieve the goals you desire?

      Maybe a few hundred people will see these blogs???
      With the TV news, maybe over one million will be made aware of Caltrain's stupidity and incompetence?

      The problems at Caltrain need to be brought out in the open, and these blogs help, but can only go so far. The power of the press/TV news must not be underestimated.

      Delete
    3. If the story did get onto the evening news, it would be made accessible to the audience, the viewers would get outraged and then wait to be outraged by something else. The story would be spread further, but thinly, and few, if any of the audience that got the message and understood it would do anything to agitate for a response/solution.
      Say what you want about my response to the problem, but don't just bash the blog. What are you doing to help? Are you going to the tv stations to get them to respond? Or just saying Clem isn't already doing enough?

      Delete
    4. The more the problem gets out, the more chances of seeing something done about it. Maybe it would catch the eye of some watchdog agency, legislative analyst, civil grand jury, GAO, after all this is taxpayer money that Caltrain is floundering.

      I have gone to NBC 11 Investigative Unit and they were interested but I could not provide the technical details they need. I figured Clem or Richard M could do that. Plus my time is limited. Maybe Roland could provide more details and time for the TV stations.

      Delete
    5. Roland is one of those people that goes to every board meeting of every agency he can find and makes comments on every agenda item. News agencies would listen to him for a sound bite, but they wouldn't hang their story on him.

      Delete
  8. I wonder how many (if any at all) are attending the UIC congress in Brussels, which as of today has a main track about ERTMS and its implementation.

    And ironically, there is also the Railway Forum Berlin going on with a track "digital railway"; similarly interesting and important.

    ReplyDelete
  9. If the contractor can deliver the system, why is Caltrain on the hook for the costs to get it to work?

    What type of contract is this?

    ReplyDelete
    Replies
    1. er, "can't", as in I can't spell

      Delete