02 August 2020

PCEP: Farce Majeure

This post serves as a place to track monthly status updates of the Peninsula Corridor Electrification Program, peeling back the rosy pronouncements put forth by the managers of this deeply troubled project. Let's start off with our handy foundation & pole progress tracker, updated monthly

Monthly updates will be added here as they occur.
For a northbound cab view of the corridor construction as it stood on July 18th, 2020, see this video by YouTube user Flat Train.
Notes from the August 2020 PCEP progress report
Another month, another slip. Electrification substantial completion is delayed by a month to 3/26/22, for a nine-month slip over the last 18 months. That's just a milestone, and the tasks leading up to it are even more dramatically delayed, with Segment 2 OCS slipping by a whopping seven months! All the electrification tasks are now jammed up against the extremely compressed segment testing, itself slipping and pushing out integrating testing and pre-revenue service by a month. To prevent electrification from exploding onto the primary critical path, heroic schedulers have cut down phased revenue service by a month. The real story here is the secondary critical path, which runs through delayed signaling installation, testing and cut-over activities... and yet these bars are colored a soothing shade of green. Look for these to blow up very soon, with a 3-month slip having just occurred in Segment 4, the one furthest along.

Foundation production continues to flounder at 49 for the month versus 161 promised. The laughable end-of-year completion milestone has slipped by 3 months, allowing the absurdly high future production rates to drop to the merely never-achieved value of 168/month. Six foundations appear to have been "unbuilt" since last month, with completed totals dropping in Segment 2 Work Areas 4 and 5. This may be a bookkeeping error, so the graph shows 1940 completed versus 1934 in the report. Overall, foundation installation is still trending towards completion in late 2021.
Notes from the July 2020 PCEP progress report
This month, as expected, a mere 40 foundations were installed versus a promise of 186. The promised numbers keep going up to maintain the pretense of finishing before year's end, with 299 foundations/month promised in October and November, over seven times the actual July rate. Detailed accounting is slightly complicated by the fresh inclusion this month of 86 foundations previously constructed outside PCEP scope for the South San Francisco and Hillsdale projects. Since the report doesn't state when these were completed, we spread them out over Jan-Jun 2020. Extrapolating at the current 3-month trailing average production rate, foundations will be completed in November 2021, almost a year behind the advertised schedule. The long-promised acceleration of foundation production is not reflected in monthly actual totals for 2020, which casts doubt on whether such an acceleration will ever materialize.

Budget burn rates for the various contracts are also consistent with a one-year delay, and that is before any pandemic impacts. The dashboards show an overall deceleration of spending, with the past 3 month average burn rate trailing the past 12 month average. Burn rate would now have to double to finish on time, which is plain to see just won't happen.

In the Appendix C schedule, there is a 7-month slip for traction power in Segment 1, and smaller slips in all the other segments. These slips remove all the slack that remained before electrification becomes the critical path. It's now a horse race (or snail race?) between Stadler and BBII.

Expect your first EMU ride in mid-2023.

Notes from the June 2020 PCEP progress report

The pandemic and the words "force majeure" are starting to make a more prominent appearance in the report, providing useful cover for Balfour Beatty's woeful schedule performance. A pandemic-related day-for-day slip at Stadler continues to provide cover, under the theory that the schedule critical path still runs through EMU production-- a condition that remains true on paper only because the secondary critical path has been slashed to the bone by unreasonably compressing key testing and integration tasks at the very end of the program.

The dashboards in section 2.1 don't lie: to finish on time, Balfour would have to triple their burn rate from $5.7 million/month to $17.2 million/month. Overall, project spending is about $700 million behind plan, indicative of severe schedule under-performance. At current burn rates, PCEP will finish no earlier than mid-2023, close to a year behind the dates currently being promised.

Foundation production for June was promised 71 / actual 105, a rare over-performance. Now do July, when an unprecedented 186 foundations were promised. As of this report, the foundations are 56% installed, and poles are 44% installed.

As of this writing in August, 2020, none of the quarterly FTA PMOC oversight reports for 2020 have been posted by Caltrain. These usually provide an unsparing look at the internal challenges of the program, but with election season approaching there is surely a rising incentive to keep them out of the public eye.

Notes from the May 2020 PCEP progress report

Foundation production, despite the insistent promises of past months, has crashed back to the dismal level of 44/month. Undeterred, project managers project ever higher and unachievable future rates (nearly 300 foundations are planned for November) in order to finish within the current calendar year.

For the EMUs, a new change order was approved to defer the installation of interior wheelchair lifts, the final nail in the coffin of the high/low boarding solution. Platform interface-wise, the EMUs will now be configured exactly the same way as the existing Bombardier cars. While recent photos from Salt Lake City show the upper doors installed, these will soon be removed and replaced by plug panels.

In a bit of good news, the regulatory compliance documentation for EMU crashworthiness has been approved by FRA, which is no small feat. One hopes sufficient spares of fiberglass front cladding have been ordered to withstand the usual grade crossing carnage.

The pandemic has delayed testing of the first trainset in Salt Lake City, such that its trip to Pueblo, Colorado for dynamic testing is delayed to November and slipping day for day.

The milestone schedule has slipped again, with electrification substantial completion delayed to 2/26/2022, a slip of 8 months since late 2018. Revenue service has slipped 2.5 months to late July 2022, all but eliminating the margin against FTA's deadline of August 2022. The pandemic will surely be invoked to delay the deadline.

Stadler is still claimed to be on the critical path, now with a convenient day-for-day pandemic slip that provides a welcome fig leaf to the Balfour Beatty electrification work.

The Appendix C schedule finally shows signal construction work. Notably, this work has pushed out the testing of segments 1, 2 and 3 by up to 8 months, with compressed testing tasks taking place at the end of 2021. The testing of the entire electrification system has been compressed from ~6 months to less than 3 months. Pre-revenue testing has been further curtailed to six weeks. There is no discussion or justification of this extremely sporty schedule compression, other than it maintains the illusion that the critical path runs through Stadler.

In the risk list, three new risks have appeared to justify what is surely the consequence of Buy America procurement for the EMUs: quality issues, failed factory tests, and poor integration and control of new U.S. suppliers. These seem to be clear and present issues, rather than risks.

Notes from the April 2020 PCEP progress report

Foundation installation recovered a bit, and an explicit (if likely unachievable) plan was published for how many foundations would have to be completed in each of the remaining months of 2020 in order to finish within the year.

Shipping the first train to Colorado (for high-speed testing) continues to be delayed. This is an important "schedule hold point" where contingency budgets are re-evaluated, and we are now 14 months into a 19-month gap that has opened in the sequence of schedule hold points.

Speaking of contingency, $32 million of it was used this month alone, of which $25 million was shoveled over to PG&E for interconnection work. Why was the contingency budget not replenished by the amount not paid to the party formerly on the hook to perform the work?

New risks: #321 if PG&E makes trouble about the single-phase loading of their substations, then the system cannot be energized. #322 if substations aren't completed on time to get powered up, then testing will be delayed. And then the kicker: #323 "FRA concerns require redesign".... don't leave us hanging, be specific!

Finally, it's the beginning of June and none of the FTA PMOC reports for 2020 have yet showed up. Who is slow-walking these important oversight documents, the FTA or Caltrain?

Notes from the March 2020 PCEP progress report

Foundation installation continues to fall hopelessly behind. The average total for the entire first quarter of 2020 was eight foundations per month (that's right, you can count them on two hands!) and if that rate is sustained, all foundations should be complete by the year 2036. Of course, the report promises a significant acceleration, but the stated goal of completing another 1544 foundations within nine months to support the end-of-year foundation completion milestone has gone from ridiculous to downright laughable. The board and public should be insulted by such a dishonest status report, insisting that everything is on schedule. It's okay to be late, but it's not okay to be so nakedly dishonest about it.

Notes from the February 2020 PCEP progress report

1) Foundation production for February is again ZERO, despite repeated affirmations throughout the report that there is a schedule to finish everything by the end of this year. The required average production rate to reach this goal is 157/month (excluding foundations that are part of SSF and 25th Ave projects); this is higher than the all-time record of 151 set in November 2019. The likelihood of missing the end-of-year target is darn near one hundred percent.

2) The Appendix C schedule shows continuing month-for-month slips in the OCS and traction power tasks, with the selective exception of the segment 1 OCS task-- which if delayed would push the BBII work onto the critical path of the project. To avoid this, the task duration was shortened, using a well-known scheduling trick.

3) delivery of trainsets 2 and 3 is delayed nine months and six months, respectively. That sure is a long time to retrofit flip-up seats. Is there something else we aren't being told?

Notes from the January 2020 PCEP progress report

1) foundation production is at ZERO for the month, with the rate required to complete by the end of the year having increased from 131/month to 143/month. The stated reason for zero foundations is because the contractor "did not have the rebar cages", of which enormous stacks can plainly be observed rusting away at Burlingame, Redwood Junction, and possibly other locations. Something big has come up and Caltrain isn't being transparent about it.

2) Schedule milestones are said not to have budged, despite the latest FTA PMOC report (December 2019) stating that the contractor's schedule shows a substantial completion date of January 2024. That's right, twenty-twenty-FOUR.

3) The flip-up seats that will be added to the bike cars are the subject of a change order that costs $1.96 million, to buy 4 flip-up seats x 2 bike cars x 19 trainsets = $12,900 per flip-up seat. No word on what material these are made of, but solid gold is not out of the question.

4) The signal modifications and grade crossing Constant Warning Time tasks that underlie the contractor's major schedule slips still do not appear on Caltrain's tracking schedule. It's harder to track the progress of a task when it isn't even on your schedule.

5) The appendix C schedule shows a wave breaking in EMU deliveries, with early deliveries delayed by ~3 months and later-produced trainsets being delivered before the earlier-produced trainsets. Must be those flip up seats and door plug retrofits.

Notes from the December 2019 PCEP progress report

1) foundation production has faltered again. The goal posts stayed put this month, but the production rate required to complete by the end of this year has increased from 124/month to 131/month. This month: just 44.

2) appendix C schedule shows a large slip in SCADA (six months!) leaving just 1 month of slack before pre-revenue testing begins. This is shaping up to be yet another secondary critical path. Meanwhile, the completion of traction power construction in segments 1, 2 and 3 is in a month-for-month slip even after the large schedule slips recorded in last month's update. The tsunami buildup continues.

On the good news front: production photos posted on calmod.com appear to show that the door to the EMU cab compartment will have a railfan window affording a view into the cab and out the front of the train. Train nerds rejoice!

Notes from the November 2019 PCEP progress report

1) foundation production has accelerated to a record monthly total of 151, but the goalpost for target monthly average has moved again from 8/31/2020 out to 12/31/2020 (four months). For the old target of 8/31/2020, the required monthly productivity would have been 179 foundations/month. With the newly relaxed milestone it is 124 foundations/month.

2) Appendix C schedule continues to show "tsunami buildup" where a wave of delayed tasks compresses against an artificially held RSD milestone. Most notably, electrification system testing (schedule line 41) has compressed from 222 days to 183 days (18% shorter) and phased revenue service (schedule line 83) has compressed from 90 days to 69 days (23% shorter).

3) The date when you will be able to board an EMU as a passenger for the first time (i.e. the beginning of phased revenue service) has slipped by a month to February 1st, 2022.

4) While the critical path is still stated to go through vehicle manufacturing, ten EMUs will have been delivered by the start of phased revenue service. Is ten enough to begin phased revenue service? If so, EMU manufacturing isn't your critical path.

As observed with last month's notes, Caltrain is making increasingly desperate schedule modifications to maintain the appearance that electrification is not on the primary critical path. With reality biting, it is doubtful they will be able to keep this up for more than a couple of months longer. Expect fireworks by March or April 2020 board meeting.

Speaking of fireworks, Happy New Year 2020 to transit nerds everywhere!

Notes from the October 2019 PCEP progress report

1) Figure 2-5 (foundation production) shows a monthly target for the production rate required to meet the schedule. This monthly target has been stuck at 174 since they started publishing this metric, which is an error in whatever spreadsheet they are using to make this chart. The correctly calculated numbers for the last 5 months (foundations-to-go divided by months left) are: 174, 178, 191, 198, 221. In this latest report they moved the goalpost from 6/30/2020 to 8/31/2020, which bought them an extra two months but used up the schedule slack. By that metric, we're back to 1766 to go divided by 10 months = 177. Hopefully this error will be corrected in future reports.

2) The contractor has never reached 177 foundations/month. To date the record is November 2019, reportedly at 151. (Interestingly, even this record would further bump up the rate to complete from 177 to 179.) This figure of 179 would have to be sustained without interruption until completion. Given that on average, the more difficult foundations (where conflicts are found with existing utilities such as Caltrain's very own PTC fiber optic cables) are being delayed and left to be addressed later than the low hanging fruit, it will become increasingly difficult to maintain rate 179.

3) In the Appendix C schedule, OCS completion has just slipped by one month for three out of the four segments. OCS completion in segment 1 (San Francisco) is now on a secondary critical path, followed immediately by segment testing and system testing. The only reason this didn't become the primary critical path this month is that they compressed system testing by one month, holding the end of system testing at 12/31/21. Compression of testing periods is a red flag.

4) In the Appendix C schedule, the logic is constructed such that it is necessary to have 14 EMUs on hand by the end of "phased revenue testing" which means service is operated with a mix of diesels and EMUs. This is what makes the critical path go through EMU production. In reality, what is most important is the *beginning* of phased revenue testing, which is when you will be able to board an EMU for the first time. Right now this milestone is at 1/3/2022 and has zero slack (i.e. it is on the critical path).

5) The latest PMOC report (September 2019) reveals that the contractor's working schedule (so far rejected by Caltrain for various reasons) predicts substantial completion of electrification on 7/4/2022, six months later than carried in the Appendix C schedule or 12/31/2021.

I expect Caltrain to make increasingly desperate modifications to the program schedule, including further compression of the system test period, to maintain for as long as possible the appearance that electrification is not on the primary critical path. Let's see how long they can obfuscate before finally fessing up.


  1. Why isn[t someone being fired for these patently dishonest reports?

  2. Because it's a lot easier to continue pretending that the schedule is still (more or less) achievable than to take the very public step of firing the project manager, acknowledging there are major ongoing schedule slips going back for many months and then explaining how & when the progress/productivity will be brought back on track?

  3. "Something big has come up and Caltrain isn't being transparent about it."

    PG&E having something break on their end? I wouldn't rule it out. Imagine the literal fireworks if Caltrain goes to energize and PG&E has a transformer blow up, start a huge fire and kill people. It's certainly plausible. Alternatively, it's conceivable that Caltrain's power system could somehow interfere with signalling and cause a major problem there. Or both. This IS the first RR 25kV system being built in CA, everything from dump loads not working right to temperature control has to be worked out all over again. If we are to read tea leaves, we should look at Denver's experience for this and compare it against Pennsylvania's.

    As for fessing up, obviously that occurs November 4th. Either the tax passes and Caltrain blames Covid for the delays (which I personally don't consider unreasonable... who knows what the hell is happening in the field if people don't show up) or the tax doesn't pass and Caltrain belt tightens.

    There's also the Samtrans Classic™: mixing the foundation concrete wrong and having to re-do it. Probably why Caltrain is now getting an independent auditor.

    1. The issue from the PG&E end is that they tend to prefer that their 3 phase system be evenly loaded, but railroad electrification with single phase AC is inherently a large unbalanced load since the primary of the transformer in the substation is wired across 2 of 3 phases. Generally it's not a big enough of a problem to cause issues for the grid at large.

    2. "This IS the first RR 25kV system being built in CA, everything from dump loads not working right to temperature control has to be worked out *all* *over* *again*." [[ emphasis added ]]

      ... that's the problem, right there.

    3. So they have a 3 phase system. Could they power 1/3 of the system from each phase? I understand that phase breaks do exist in RR electrification for when trains cross between unsynchronized power grids. Or does this basically amount to adding extra complexity where it's not necessary? Is PG&E's system synchronized throughout the entire state?

    4. "Could they power 1/3 of the system from each phase?"

      No, they cannot. Because the interface to PG&E is the traction stations, and they have only 2 traction substations.

    5. Modern power electronics can solve this problem. Believe it or not, but rectifiying 3 phase AC into DC, inverting the DC into high frequency AC, transforming the high frequency AC, rectifying the high frequency AC into DC and finally inverting the DC into 1 phase AC is even more efficient then transforming 1 of the 3 phase directly.
      Added bonus is that you are in full control of the phase on your overhead wire, so you could eliminate phase breaks.

  4. Has anyone outside Caltrans and the FTA ever seen the PMOC reports? They should be available as public records, though I'm not surprised they're not posted to a website, I'm not aware of any agency that posts them, no matter how well or poorly the project is progressing.

  5. August 2020 PCEP update is up.

  6. Anyone care to guess at founddations completed in the September report?

    (And why are these reports ~60 days after the beginning of the reported month?)