Formulation of a Service Quality Metric

The quantitative formulation of an overall quality metric, which can be extracted from an arbitrary timetable, is necessary to objectively answer the question “is proposed timetable A better than proposed timetable B?”

Such metrics facilitate the trade-study and optimization process of planning a new timetable, and must take into account several factors, including not just the quality of the service provided to passengers but also other factors that passengers don’t think about, such as robustness to disruption, fleet size and crew time considerations.

For today, however, we will focus exclusively on quantifying the quality of the service provided to passengers.  This particular formulation proceeds in eight reasonably simple steps, pulling together earlier information on timetable metrics and demographics.  It is only one example of how one might formulate a service quality metric, something that Caltrain has never explicitly done and could benefit greatly from doing as they share the pros and cons of various blended service plans.  This is one way to do it; what's theirs?

Step 1: Extract trip time and wait time statistics for each origin and destination pair.  By straightforward analysis of the timetable, one can figure all the possible trips between any origin station A and destination station B (including transfers) during a one-hour span during the morning peak.  One can then determine (in units of time):

1. The average trip time between A and B (Tmean_AB)
2. The fastest trip time between A and B (Tmin_AB)
3. The average wait between trips that connect A and B (Wmean_AB)
4. The longest wait between trips that connect A and B (Wmax_AB)

The first two metrics measure trip time on board the train, and the next two can be used as a proxy for measuring typical wait times on the platform.  The trip time and wait time figures are intrinsic to the timetable and can be extracted by a computer program.

Step 2: Compute an “effective” trip time from A to B by computing a weighted sum of the time components extracted above. This is where judgment calls start to be made. Taking into account the waiting times Wmean and Wmax is just as important as the actual trip times Tmin and Tmean, in order to properly account for the frequency of service. For example, the effective trip time could be defined as:

Teff_AB = (30% of Tmin_AB + 70% of Tmean_AB) + (20% of Wmean_AB + 15% of Wmax_AB)

The trip time term (30% of Tmin_AB + 70% of Tmean_AB) accounts for some trips being shortened by express service. The waiting time term (20% of Wmean_AB + 15% of Wmax_AB) properly penalizes long service gaps, but remains shorter than the waiting time incurred when the passenger shows up randomly, which is 50% of Wmean_AB.  This lower weighting reflects the fact that passengers don’t show up randomly, but usually time their arrival at origin A for a particular trip to destination B.  For example, when trips are available every 15 minutes, the waiting term works out to a quite reasonable 5 minutes. The effective trip time is a reasonably good measure of how long it will take you to get from A to B.

Step 3: Determine the “effective” speed between origin A and origin B. This is simply distance divided by time, or: V_AB = d_AB / Teff_AB where d_AB is the distance between A and B. This process is repeated for every origin and destination pair A-B, and describes not the speed of a train, but the average speed of a typical trip from A to B including waiting time, based only on the available service provided by the specific timetable being considered.

Step 4: Compute weighting by population and jobs.  This is where census data enters the calculation, as it must.  For the morning rush hour, since ridership consists primarily of people going from their home near A to their work near B, we calculate a potential ridership weight based on how many people live near A and how many people work near B.  This simply reflects that if a lot of people live near A and work near B, it is more important to provide fast service between A and B than between other station pairs where fewer people and jobs are located.

The “home weight” Whome_A of origin station A is a simple gravity sum (1/r squared law) of the residential population, taken from the 2010 census, as described previously in greater detail.  Each person is divided by the square of how far they live from station A, to reflect that people who live further away from the station are less likely to use it. To prevent over-counting people who live very close to the station (where the 1/r squared term diverges), anyone living closer than ¼ mile from the station is considered to live ¼ mile away.  The resulting weights are shown at left, in orange.

Similarly, the “work weight” Wwork_B of destination station B is a simple gravity sum of the number of jobs over $40k, again taken from census data. Each job is divided by the square of how far it is from station B, to reflect that people who work further away from the station are less likely to use it. Once again, to prevent over-counting jobs located very close to the station, any job closer than ¼ mile from the station is considered ¼ mile away.  The resulting weights are shown at right, in blue.

Step 5: Compute weighting by distance. Regardless of where people live and work, there are upper and lower limits to how far they will typically commute by rail. Extremely short trips are less likely because of the overhead of access and egress to and from the station at each end of the journey. Conversely, extremely long trips are less likely because of their sheer duration.  As it turns out, the typical rush hour trip on Caltrain turns out to be about 25 miles, or 40 km.

For our purposes, the distance weighting is constructed by drawing a curve with a peak at 40 km. This distance weight starts off at zero for a trip distance of less than 7 km (reflecting no demand for such short trips), peaks at a distance of 40 km, and decays slowly thereafter.  Converted to miles, it looks like the figure at left.  The underlying math to draw this curve is a Rayleigh distribution with a peak at (d-7) = 33, where d is the trip distance in km.

Step 6: Combine the population, jobs and distance weights to obtain a ridership potential matrix.  The ridership potential matrix R is a matrix of size N squared, where N is the number of stations.  Each element R_AB of this matrix represents the "potential" ridership (in arbitrary relative units) that can be tapped into during the morning commute from origin A to destination B.  This ridership potential matrix has an important property: it is independent of any timetable, and concisely describes the underlying demand that inherently exists out there--regardless of how or whether that demand is met by rail service.  Each element R_AB is given by the product:

R_AB = Whome_A * Wwork_B * Wdistance_AB

Note that the matrix R is not symmetric, because the number of residents and jobs near each station differs.  For example, far more people will want to commute to SF Transbay in the morning than from it, since the number of jobs within a half mile of that station is greater than all the jobs within a half mile of every other Caltrain station all the way to Gilroy combined.

Step 7: Compute the service quality matrix. The service quality matrix Q is again a matrix of size N squared, where N is the number of stations. Each element Q_AB of this matrix represents the quality of morning rush hour service from station A to station B, and is given by the following formula:

Q_AB = R_AB * V_AB

This combines R_AB, the timetable-independent ridership potential from origin A to destination B, with V_AB, the timetable-dependent effective speed from A to B.  If you have a preferred AM origin and destination (as most commuters do), then you can compare your Q_AB for various timetables to see how any given timetable will meet your own specific needs.

Step 8: Extract overall service quality scores. The service quality metrics must be bench marked against some reference, so they are simply normalized against the most current timetable.  That means today's timetable will score 100, by definition.  By adding the elements of Q over all possible origin and destination pairs, we can quantify the degree of service improvement and compute a score for the entire timetable as well as a score for each individual station. The overall timetable service quality score is S = ΣQ / Sref, i.e. the sum of all the elements of Q divided by the corresponding sum for today's timetable.

An entire timetable can now be distilled to its essence, a single service quality score.

We are now empowered to compare various timetables and understand quantitatively the pros and cons of each.  This method will tell you objectively whether timetable A provides better overall service than timetable B--and if you happened to disagree with the scoring outcome, then your argument would be with the scoring method and not any particular detail of this or that proposed timetable.  Beyond the mathematical minutiae of the rather simple scoring method presented here, the larger point is that there needs to be a defined scoring process and a framework for stakeholders to discuss what makes a good timetable.  This scoring process is absolutely essential for planning future blended service on the peninsula.  Caltrain's approach so far has been to prescribe a certain skip-stop pattern (see Tables 7 and 8) and restrict all analysis to that particular pattern, seemingly without regard to overall service quality!

CBO$$

Today another $40 million was pumped into Caltrain's CBOSS project, a unique reinvention of the wheel that is nearly certain to fail based on the past track record of similar projects in the Bay Area.

The cheerful press release contains the following gem:

The system, which costs $231 million, is fully interoperable with freight traffic using the Caltrain corridor and future High Speed Rail trains.

This one sentence could easily make one choke twice on the same pretzel.  First, the astronomical cost of more than $4 million per route mile is more than triple the world standard for similar projects.  Second, the claim of interoperability with HSR is an outright lie according to Caltrain's own contracting documents.

Interoperability, in this context, means spending additional millions to outfit each and every high-speed train in California (in a fleet that will dwarf Caltrain's) with CBOSS on-board signaling equipment.  Under this definition, CBOSS is no more interoperable with future "High Speed Rail trains" than it is interoperable with the red rail-cycle pictured above.

HSR and Grade Crossings

The Palo Alto Daily Post, bastion of journalistic integrity, has on several occasions reported as fact that the blended HSR system on the peninsula would require 100% grade separation in order to share tracks with Caltrain, resulting in dozens of seized residential properties.

Not so.

When sharing tracks with other trains, high-speed trains can and do use grade crossings on a daily basis, with all their attendant risks.  Examples of this practice abound in Europe, where new HSR networks have been patched into existing rail networks.  High-speed trains are limited to the same speeds as other trains when using grade crossings, and are exposed to the same collision risk.  The trains are built to take it (the relevant standard is EN 15227) and have been involved in dozens if not hundreds of grade crossing accidents over the past three decades.  When a train collides with a car, damage to the train is usually only cosmetic.  But high-speed trains have also collided with trucks and farm tractors, with more dire results, but only one known passenger fatality in 1988.  A small sample of those horrors is provided at right.

On a mostly grade-separated corridor like the peninsula, new grade separations are desirable primarily because they reduce gate down-time and speed the flow of road traffic, and secondarily because they reduce the risk of collision with pedestrians, cars and trucks.  They are not inherently required to operate high-speed trains in a blended system, any more than they are required to operate Caltrain.  Suggesting otherwise amounts to baseless fear-mongering that is best confined to the editorial page.

Now What?

In an historic vote, the legislature today approved a funding package worth about $8 billion to begin construction of the first high-speed rail system in the Americas.  To make the package politically more palatable around the state, it included the immediate release of $706 million of so-called "book end" funding in the form of Proposition 1A bonds specifically allocated to the modernization of Caltrain, per the recent Memorandum of Understanding approved by all involved agencies.  The $706M total includes $600M of high-speed rail funding and $106M of non-HSR connectivity funding, from pots of money that are subject to different constraints.  These sums form the lion's share of a $1.456 billion funding package that covers both electrification and a new signal system for the peninsula rail corridor.

While this is no doubt a landmark occasion to celebrate for supporters of modern rail transportation, today's vote will probably not cause anything dramatic to happen on the peninsula in the short term.  Consider:

Taxpayer Lawsuits.  The opposition remains fervent and relentless, and a lawsuit challenging the release of $600M of HSR bonds to improve the Caltrain commuter rail system, with not a high-speed train in sight for more than a decade, is a near certainty.  Protections are built into the law, and require several conditions to be met for release of the funds.  Approval by the legislature is only one of those conditions, and the interpretation of the remaining ones is likely to become legally contentious.

The Environmental Clearance Process.  In April of 2010, Caltrain's electrification project had already obtained federal environmental clearance under the National Environmental Protection Act (NEPA) as the Caltrain board of directors came within a few dramatic minutes of certifying the Final EIR under California's Environmental Quality Act, or CEQA.  The board stopped short, under a surprise threat of a CEQA lawsuit, preferring to resolve any issues outside of the legal system before certifying the FEIR.  While the scope of the electrification project has not changed under the recently approved MOU, the project has now become the first in a series of incremental investments leading up to the "blended system" envisioned in the latest HSR business plan.  That means the electrification EIR may go back to square one for yet another round of public circulation (following prior rounds in 2004 and 2009), a process that is likely to take several years.  It would be surprising to see a new FEIR before 2014.

CEQA Lawsuits.  The sole enforcement mechanism built into CEQA is the lawsuit; it is therefore expected that lawsuits could follow the certification of any EIR.  While clearing electrification as a stand-alone project might be legal under CEQA, the issue is complicated by the project's new association to high-speed rail.  The two-tiered environmental clearance process adopted by the HSR project has already run into serious resistance, with the Bay Area to Central Valley Program EIR about to enter its third round of litigation since 2008.  HSR opponents could easily argue that funding the electrification project under Proposition 1A requires the prior clearance of both this program EIR as well as the project-level EIR for the "blended" San Francisco - San Jose section of the HSR project, including all the project phases expected to be completed after electrification.  Those later phases would include more controversial measures such as the construction of new overtake tracks and new grade separations.  This document is yet to be drafted; while the peninsula project EIR for the full-bore four-track system (still allowed for in the program EIR) was almost ready to circulate as of late 2011, it will require extensive revisions before it conforms to the "blended" configuration.  And that's before it becomes mired in what could become years of CEQA litigation.

The Lead Agency Issue.  The high-speed rail authority has in the past been openly hostile to funding Caltrain improvements.  The new leadership, under board chair Dan Richard, may not be much different.  Richard, like Kopp before him, is a longtime supporter of the expansion of BART, which has always been in invisible tension with Caltrain enhancements.  While he has extolled the merits of the blended book-end approach to gain political support for the entire HSR endeavor, this stance could very well weaken now that the legislative hurdle is passed.  Prior to the vote, he was quoted as saying "The Legislature wanted to emphasize that this money would be there for (the Bay Area and Southern California). And they’re right," highlighting that it is the legislature pushing this funding, not the CHSRA.  Indeed, the Authority, and the transit industrial complex behind it, may be reluctant to push for the peninsula improvements (a) because the opposition there is intense, (b) because of inter-agency rivalry with Caltrain, and (c) because the proposed projects do not involve large-scale civil works of the sort that Parsons Brinckerhoff likes to design, and its acolytes in the construction industry like to build.  Progress on the electrification project could thus depend on which agency leads the EIR process and pushes the project to fruition.  Caltrain is both competent and motivated, but the CHSRA could easily drag its feet--after all, the legislature has only authorized the bond funds to be spent, but the CHSRA retains full authority over when to actually spend them.  All the MOU demands of them is "good faith," which has been in demonstrably short supply.

CBOSS.  While the spotlight is on the electrification project, the MOU and newly passed HSR funding also covers Caltrain's new Advanced Signal System, also known as the Communications Based Overlay Signal System or CBOSS, and often criticized on this blog.  This project is a necessary pre-condition for the operation of light-weight European-style trains, and must be completed by the end of 2015 under a federal mandate that shows signs of being delayed to 2018 or 2020.  Despite Caltrain's repeated insistence to the contrary, CBOSS will not be compatible with HSR other than by fitting two separate, expensive, and functionally redundant signaling systems on high-speed trains, an unavoidable and inconvenient truth that may call into question the wisdom of spending even one cent of HSR money on CBOSS.  A far better outcome would be to make the peninsula rail corridor a testbed for the actual train control system to be deployed on the HSR system, based on the increasingly mature worldwide ERTMS rail signaling standard.

The Timeline.  The money is available only until June 30th, 2018.

UPDATE: The Poison Pill.  At any time before then, a single stroke of the pen from the Department of Finance can transfer the money to the Central Valley projects, per the Budget Act of 2012, Section 2.00, Item 2660-104-6043, Section 3, Provision 2.

The legislature's momentous step leaves many questions unanswered.

• Is the funding of Caltrain improvements using high-speed rail bonds legal?
• Will opponents hog-tie the electrification EIR to the high-speed rail EIRs in a bid to delay?
• Can the existing electrification EIR be tweaked, or is it back to square one?
• How eagerly will the CHSRA push electrification forward, if the focus is initially in the Central Valley?
• Is it legal to spend HSR money on CBOSS?
• Will the project be shovel-ready by June 30th, 2018?
• How will questions of leadership be resolved, among Caltrain, the CHSRA, Parsons Brinckerhoff, and the regional design consultants?
• Will the agencies finally treat technical compatibility between Caltrain and HSR, as long advocated in these pages, as the priority that it ought to be, allowing any train to use any track to serve any platform?

Only one thing is sure, there is a lot more sausage-making still ahead of us.

Is Demand-Based Planning a Myth?

Original photo by qviri

With over 80% of riders using Caltrain to commute to their jobs during rush hour, one would think that the service would be planned around where people live and where people work, using cold hard numbers from the census.  That's known as traditional demand-based planning: provide service where the most people will use it.  It's not rocket science, and demand-based planning is used all around the world to plan excellent rail service.

But not here on the peninsula.

In a contrarian argument made circa 2005, Caltrain's operations staff claimed that demand-based planning is a myth. (14 Mb PDF file)   At the time, Caltrain was crowing to its industry peers about the success of the Baby Bullet.  The keys to success included "Questioning Traditional Planning Processes" and "Trusting Your Intuition".  Numbers don't matter, just go with your gut!

In the years since, there has been plenty of hard evidence that the Baby Bullet has severely reduced ridership at many locations, especially in Santa Clara County.  Indeed, data from the 2010 census can be correlated to the latest Caltrain ridership data without ever looking at a timetable to reflect quite accurately which Caltrain stops are under-served and falling short of their ridership potential.

Maybe demand-based planning isn't such a myth after all.  Maybe numbers don't lie.  Here's hoping that objective, quantitative metrics will play a central role in planning future blended operation scenarios with high-speed rail.  This stuff is too important to trust anybody's intuition.