San Jose Done Right

Map of VTA's BART extension

San Jose is the tenth largest city in the U.S. (by population), with more people than San Francisco; the city achieves this statistical feat by encompassing 180 square miles.  Such a large and populous city surely deserves top-notch rail transit.  BART is widely viewed as top-notch rail transit, which is why the city and VTA (Santa Clara County's transportation authority) have made extending BART through San Jose their very top priority.

Actual expenditures from VTA
Measure A (2000) sales tax, 2015

So overwhelming is the priority for BART that VTA re-programmed the revenue from a half-cent transit sales tax (Measure A) passed back in 2000 primarily to the BART extension, breaking a promise made to voters that a significant portion would fund Caltrain electrification.  The actual expenditures through 2015 are shown in the diagram at left; money spent on the BART extension is shown in blue, and money spent on Caltrain in red.

As can be readily observed, the Measure A money is nearly gone, and the BART tunnel through San Jose is not even started.  That's why another half-cent transportation sales tax Measure B was passed in Santa Clara County in November 2016 to raise a further $6 billion through the year 2047.  Exactly like 2000 Measure A, 2016 Measure B promises lots of funding for Caltrain, an ample 16% slice that includes grade separations ($700M) and capacity improvements ($314M).  The small print, however, allows the VTA board to re-program the funding as it sees fit, adapting spending to "unforeseen" circumstances such as, perish the thought, an over-budget BART extension.

With San Jose and VTA suffering from a severe case of BART tunnel vision, it's important to take a more holistic view of what it means to provide the residents and workers of San Jose with a top-notch rail transit network.

San Jose Pan-Galactic Inter-Dimensional Station

San Jose planners will insist that creating a network is their highest priority, and to that effect, their Diridon Station Area Plan seeks to establish a new "Grand Central of the West," as described in Section 2.5 of the plan:

San José Diridon Station will be the best connected transportation hub on the West Coast with the convergence of virtually every mode of public transportation. Activity will increase dramatically with the addition of high speed rail and the extension of Bay Area Rapid Transit (BART) to Diridon station, combined with significant growth by current intercity rail, commuter rail, light rail and bus operators. These new services and growth in demand will create the need for a significant expansion of the existing station. 

This ambitious station development plan rests on two fundamental but unstated assumptions:

1. Caltrain, ACE and Amtrak will continue to operate Diridon station as a terminus, where out of service trains are parked for extended layover periods, wasting valuable platform space as train storage.
    
2. As a result, high-speed rail will not fit within the ground-level footprint of the station, and will most likely require an entirely new elevated facility built over the existing station.

Ridership assumptions for Diridon

These two assumptions are firmly rooted in the ambitious plans of numerous rail transit agencies that prefer to avoid stepping on each other's toes.  Each agency specifies its future needs, San Jose consultants unquestioningly tally up the numbers (see figure at right), and end up prescribing a framework that demands a massive station complex to support a nearly ten-fold increase in ridership over the next twenty years.

Caltrain and high-speed rail consultants have conducted a sophisticated simulation study known as an "operational conflict analysis" that predicted an intolerable traffic jam, with peak-hour delays of nearly an hour.

The Diridon Station Area Plan and the Caltrain / HSR operational analyses are flawed for having failed to examine and question the assumptions on which they are built.  Yes, the station has enormous potential to become a thriving transportation hub, but that is precisely what makes it a very bad place to park out of service trains. Parking or laying over trains at a station platform is the railroad equivalent of parking an empty truck in the middle of a bustling loading zone, and then concluding that the loading zone fails to function adequately and must be expanded.

Trains need to hustle in and hustle out without occupying enormously valuable platform tracks. As will become clear, the simple practice of not parking trains in the worst place to park trains enables a far more efficient and affordable at-grade station configuration for San Jose that provides the same great network effect and transportation benefits for the heart of Silicon Valley, saving enormous sums that can be re-invested to achieve a much better outcome for riders and taxpayers.

Here's San Jose done right:

1) Extend Caltrain through San Jose

Falling short: census data for the Caltrain corridor in San Jose,
overlaid with Caltrain service levels for April 2017.

Viewed as a line on a map, Caltrain already runs through San Jose and beyond, with Gilroy service having started back in 1992. San Jose Diridon station (served by 92 trains/weekday) isn't a natural terminus; south of it, there are three additional stops located within San Jose city limits: Tamien (dropping from 40 to 34 trains/weekday in April 2017), Capitol (6 trains/weekday) and Blossom Hill (6 trains/weekday). Service between Diridon and Tamien is timetabled at 7 minutes, which makes for an average speed of 15 mph, and average speeds south of there hover around 30 mph. The abysmal service south of Diridon station lacks two important attributes of top-notch rail transit: speed and frequency. That's why it's fair to say that despite that line on the map, Caltrain has yet to be extended through San Jose. It's time to do it properly.

One challenge is jurisdictional, with Union Pacific owning the tracks south of milepost 52 and VTA currently holding the rights for only ten daily round trips. However, UPRR does not make intensive use of these tracks, and as a profit-making enterprise would likely be receptive to an outright transfer of ownership while retaining trackage rights to continue operating its Coast Subdivision freight service as before. This would simply extend the existing arrangement between CP Coast (milepost 44.7) and CP Lick (milepost 51.6), where the Caltrain owns the right of way and dispatches the track, southwards to CP Coyote (milepost 59.9).

Another challenge is institutional, with VTA having a vested interest in making commuters use the Santa Teresa branch of its light rail network. The Caltrain San Jose extension would parallel this line, possibly cannibalizing some of its ridership.

Built-up areas shown in black on a map
by the DLR Earth Observation Center
(Global Urban Footprint).  Tamien,
Capitol and Blossom Hill are shown
disconnected, as they are today.

Demographically, the southern half of San Jose is a rich but poorly tapped source of commuter ridership, with dense residential neighborhoods surrounding the corridor. More than 100,000 people live within two miles of the Tamien and Capitol stops, and 75,000 people live within two miles of the Blossom Hill stop. Census data argues strongly for locating the Caltrain terminus at Blossom Hill, with an electrified train storage yard / layover facility in this large vacant space [UPDATE: that large vacant space seems to be spoken for, so look for other unbuilt spaces in map at left], a far better place to park out of service trains than in the middle of San Jose Diridon. ACE and Amtrak trains could be turned at the existing Tamien layover facility.

Turning all Caltrain service at Blossom Hill would improve service for hundreds of thousands of San Jose residents and workers, at some increase in capital cost (to electrify) and operating cost (12 minute longer runs). On the other hand, it would greatly reduce Caltrain's requirement for tracks and platforms at Diridon station. Caltrain would operate through the San Jose Diridon station much like it does at the Palo Alto University Avenue station, using just two tracks and two platform faces. If that seems hard to imagine, remember that Palo Alto has almost 60% more ridership than San Jose Diridon; any perceived need for all those tracks and platforms at Diridon, and the profoundly mistaken notion of a "South Terminal", arises from existing jurisdictional boundaries and Caltrain's unhealthy habit of parking trains in the worst possible place to park trains.

2) Build high-speed rail at grade.

Thousands of cubic yards of concrete,
zero marginal transportation benefit

With Caltrain's San Jose footprint shrunk to just two platform faces, and with HSR's recent decision to shrink platform length to just 800 feet, it becomes feasible to operate the San Jose HSR service entirely within the existing at-grade footprint of the station, without the need for expensive new elevated or tunneled infrastructure. Two of the existing platforms are already over 1200 feet long and could be converted for HSR use. Just as in San Francisco Transbay, these platforms could be shared with Caltrain, taking advantage of Caltrain's new dual-boarding-height trains and leading to even more efficient utilization of the existing station footprint.

Operationally, HSR would have to quit the same nasty habit of parking trains in the worst possible place to park trains.  Trains would have to layover somewhere north of Diridon, or continue onto the peninsula rail corridor.  There is no operational need for longer station dwell times than two or three minutes within the Diridon complex: get in, board and/or alight passengers, and most promptly and importantly, get out. Go layover somewhere else than the bustling city center.

Building everything at-grade would save about a billion dollars (by foregoing about $250M for elevated approach tracks, $500M for the elevated Diridon station complex itself, and $500M for the "iconic" but entirely avoidable viaduct to cross the 87/280 freeway interchange to the south). An added benefit of the at-grade approach to San Jose is higher speeds and lower trip times. The extremely tight 1000-foot curve radius that connects Diridon to an "iconic" viaduct saps the 'H' out of HSR by limiting trains to just 50 mph, while the existing curve through the Gardner neighborhood could be grade separated and operated at 65 mph.

Rather than cower in the shadow of a new "iconic" bridge proclaiming loudly that they are just a flyover neighborhood, residents of San Jose's Gardner district would gain a grade separation at Virginia Street, improving neighborhood access that has been so brutally cut off by the I-280 and SR-87 freeways, and eliminating the sound of railroad horns--even freight train horns.

3) End the BART extension at Diridon/Arena

As planned by VTA, the BART to San Jose Phase II project doesn't just take BART to San Jose, but takes BART beyond the San Jose Diridon/Arena station, veering north to parallel the Caltrain / HSR corridor for a redundant 2.5 miles, ending in Santa Clara.  While this configuration might have made sense long ago when BART harbored ambitions to "ring the Bay" by linking Millbrae and Santa Clara, the present state of affairs argues for a different solution.

From a transportation perspective, it makes no sense to spend ~$1.5 billion of scarce transit dollars (pro-rated from the $6 billion cost of the entire Phase II project) on a 9000-foot tunnel leading to a huge Santa Clara station complex just to provide a third way to ride between San Jose Diridon and Santa Clara, two locations already well-linked by Caltrain and VTA's 522 express bus.

BART maintenance yard at Las Plumas
Avenue in San Jose, an alternative
that was withdrawn in EIR process

The main argument against truncating the BART extension revolves around a new 69-acre maintenance facility planned at Newhall Yard in Santa Clara. BART argues that Santa Clara and downtown San Jose are too far away from the nearest existing maintenance and storage facility, BART's main Hayward Maintenance Complex, to be operated efficiently. The HMC is about 21 miles from Santa Clara, requiring long non-revenue runs to stage trains to/from the end of the San Jose extension.  While this is admittedly an operationally inefficient arrangement, BART appears to have no qualms operating Phase I (to Berryessa) out of the HMC, over a distance of 14 miles. Cutting back the 2.5 miles from Diridon/Arena to Santa Clara would place the end of the line less than 19 miles from HMC, not so much further from the HMC than Berryessa already is. The HMC itself is undergoing a major expansion, with storage space for an additional 250 BART cars environmentally cleared based on a purpose and need statement that invokes servicing the BART to San Jose extension. Even then, if the HMC Phase II expansion were to prove insufficient and if maintenance and storage demands were truly that dire, a small portion of the $1.5 billion cost avoidance of truncating Santa Clara could be reinvested to provide a new BART maintenance shop at Las Plumas Avenue, an alternative that was considered during the environmental process. Trains could also be stored overnight at Diridon/Arena, to avoid long non-revenue runs at the start and end of the day. The bottom line: the argument that a Newhall shop is a non-negotiable, vital component of the BART to Silicon Valley project is technically unfounded and rests on a stay-the-course-at-all-costs logic that fails to appreciate the opportunity costs of blowing $1.5 billion on a train parking lot.

Another argument against truncating the BART extension concerns a planned airport people mover that would link Santa Clara to the SJC terminals, tunneling under the runways. Using a small portion of the $1.5 billion savings of ending BART at Diridon/Arena, the people mover could run straight to Diridon station, without the need for tunneling under the runways, and connect not just with Caltrain and BART but directly with high-speed rail--seamlessly merging the airport and the train station.

4) Use Newhall Yard for HSR

As it turns out, there is a better use for Newhall Yard than BART storage and maintenance, namely, HSR storage and maintenance.

As previously mentioned, long non-revenue runs to stage trains to/from their terminus are operationally inefficient, but BART can get by because nobody else uses their tracks. If the HSR storage and maintenance yard were to be located in Brisbane, these non-revenue runs would consume scarce and valuable operating slots on the extremely constrained peninsula corridor "blended system," further compromising service quality for all rail passengers.

A better plan is to have only a small storage / layover yard in Brisbane, with a larger facility perfectly located just north of San Jose Diridon at Newhall Yard, which would allow a portion of the HSR service to originate / terminate in San Jose without gumming up the peninsula rail corridor. Recall the blended system will be limited to 4 trains per hour per direction unless long stretches of the peninsula corridor are expanded to four tracks, an idea that faces twin obstacles of funding and community opposition.

Organisation vor Elektronik vor Beton

In Germanic countries, there is a guiding principle in rail system design known as Organisation vor Elektronik vor Beton, or roughly, organization before systems before concrete. It gives the order of priorities for quickly and affordably increasing train traffic: first you re-plan your operations, and if that doesn't cut it you improve your technology, for example by using shorter signal blocks, and only as a last resort do you pour concrete.

What is about to happen in San Jose is the exact opposite: legions of consultants primarily from a civil engineering background are (surprise!) recommending a concrete-intensive solution to a problem that is ill-posed because it hasn't first been attacked from the standpoint of re-planning train operations. The entire edifice is built on the nasty habit of parking trains in the worst possible place you could think of to park trains.

The planners and engineers working on the future of the Diridon Station area need to be sent back to the drawing board with new operating assumptions:

1. Turn all Caltrains at Blossom Hill, operating San Jose Diridon as just another intermediate stop
2. Turn all legacy diesel trains (ACE, Amtrak) at Tamien, away from the bustle
3. Turn all high-speed trains originating or terminating in San Jose at Newhall Yard

Thinking of San Jose as a terminal is misleading. The litmus test is really simple: if your timetable, operating plan or simulation has any train spending more than two minutes dwelling at a platform in the San Jose Diridon station, then it is probably flawed. Don't turn trains at the choke point of your system, so that we don't spend billions on fancy train parking with zero value to the traveling public and negative value to the taxpayer.

The Bay Area can ill afford transit mega-projects of low utility, such as the redundant BART segment beyond Diridon/Arena to Santa Clara, the giant HSR station in the sky, or more downtown train parking. The cost is outrageous, and the opportunity cost is shameful.

Optimizing the Midline Overtake

The most important piece of infrastructure required for "blending" HSR with Caltrain on the peninsula rail corridor is an overtake facility, basically a several-mile long stretch of up to four tracks that will enable faster trains to overtake slower trains.  Caltrain studies have shown that the best performance (measured by robustness to cascading delays) can be achieved with a midline overtake  from San Mateo all the way through Redwood City.  Preliminary engineering and environmental clearance for this infrastructure is now resuming, with the recent award of a $36 million contract by the CHSRA to engineering firm HNTB.

The baseline configuration for this overtake is described in Caltrain's blended operations analysis, and assumes rebuilt stations at Hayward Park, Hillsdale, Belmont, San Carlos and Redwood City with four tracks and outside platforms in the tried and true style of the 1930s Pennsylvania Railroad. High-speed trains would use the center pair of tracks to overtake slower trains on the outside pair of tracks, as shown below (click figure to enlarge):

For comparison, the figure also shows a better solution that ensures the highest level of punctuality for all trains using the corridor.  This optimized overtake configuration differs from the baseline configuration as follows:

• The slow trains run in the middle, so that disruptions to local commuter service (for example, when an incident blocks a track for hours) do not disrupt high-speed service when commuter trains are re-routed around the incident location.  This track configuration is known as Fast-Slow-Slow-Fast (FSSF) as opposed to the traditional SFFS.  Real world examples of FSSF can be seen in train cab videos from Sweden and Australia.
   
• All commuter stations are built with central island platforms.  This allows commuter trains to use either platform face without confusing passengers, and requires only one set of station amenities (shelters, elevators, escalators, stairs, ticket vending machines, PA systems, train arrival screens, lighting, benches, etc.) because there is only one platform.
   
• A major new interchange station at Redwood City, with four platform tracks and additional train storage sidings.  This station (described below) would serve as a transfer point for HSR, Caltrain and future Dumbarton trains, as well as non-rail transportation modes.
   
• A carefully planned future-proofed high-speed rail junction where the Dumbarton rail corridor meets the peninsula rail corridor, preparing for the inevitable arrival of passenger rail service across the Bay.

Here's how it would ideally play out.

Short Term: the San Mateo Grade Separation

Preliminary rendering of new
Hillsdale station with island platform

The next step in the decadal process of grade separating the peninsula rail corridor will soon begin in San Mateo.  A new $180 million grade separation project is in the final stages of planning for 25th Avenue (currently a grade crossing) as well as 28th and 31st Avenues (currently not connected).  Concurrently with this project, the busy Hillsdale station will be moved a bit north of its current location and turned into an island platform.

This project is caught in an interesting political bind.  There is on one hand a rush to complete it by 2019 before Caltrain's electrification project, to minimize disruptions to Caltrain service.  On the other hand, due to its strategic location, this project will form a key building block of the blended system with HSR, which still needs to be environmentally cleared.  It is a near certainty that this portion of the corridor will require four tracks to enable trains to overtake each other, but any attempt to design and build it as such is likely to run afoul of HSR opponents who will accuse the CHSRA of advancing their project through CEQA piece-mealing.

The southern San Mateo grade separation design will have to be very carefully considered to preserve the ability to add two additional tracks with as little disruption as possible.  Road underpass profiles and bridge abutments should be designed for four tracks, as should the elevated structure that will support the tracks.

The choice of an island platform configuration for the new Hillsdale station is either a sneaky way to build a wide four-track embankment in preparation for yet another new station with SFFS outside platforms, or is an excellent choice for FSSF because it allows future tracks to be added without rebuilding the station for a second time.  One hopes the station access (stairs, ramps, etc.) will be designed to allow the platform height to be raised easily from 8" to 50".  An intelligently designed San Mateo grade separation would atone for the terrible failures of the San Bruno grade separation, designed with great hostility towards higher speeds or additional tracks.

Medium Term: Redwood City HSR Station

Amsterdam Bijlmer (photo by tataAnne)
could just as well be the future
Redwood City train station.

While the CHSRA's plans for a mid-peninsula stop have been shrouded with ambiguity for several years, Redwood City stands out as a more optimal location for a new HSR station than Palo Alto or Mountain View, the other two locations in the running.  Unlike its neighbors to the south, Redwood City favors strong urban growth, has a large amount of railroad land available, and is reasonably well-connected to the existing road and transit network.  Redeveloping the antiquated but popular Sequoia Station shopping center would enable the construction of an elevated four-track station with plenty of capacity to support not just HSR but also Caltrain cross-platform connections and future Dumbarton service.

The station complex would feature two shared (HSR or Caltrain) 400-meter island platforms centered between Broadway and Brewster, easily accessible from both streets.  Bus connections would be conveniently located under the station. To the north of the platforms, a pocket turnback track would allow Dumbarton trains to reverse without fouling other traffic.  Similarly, to the south of the platforms, another pocket turnback track would allow southbound Caltrain locals to terminate in Redwood City before turning northwards again to serve the densely-spaced stations of San Mateo County, allowing Caltrain to serve more passengers with fewer trains and crews.  Thanks to the FSSF configuration, all this to-and-fro by commuter trains would stay well out of the way of HSR.

This would be a large train station and quite a tight fit (if you're curious about exactly how large and how tight, download this KML file into Google Earth to view the station footprint and track layout).  It would be a big change for Redwood City, but with a huge payoff: the tracks would no longer form a barrier through town, and the Sequoia Station shopping center would be merged with the station to form a gateway and a destination in its own right that is connected to downtown.  HSR service could make the city a very desirable location for business.  The new station could become the centerpiece of the ambitious revitalization strategy described in Redwood City's downtown precise plan.  But this idea is not without pitfalls, as the size of the station could be compared to plonking a couple of Nimitz-class aircraft carriers in the middle of town.  To use the tired slogan, it needs to be done right.

Medium Term: a New Fair Oaks Station

Approximate location of new Fair Oaks
station island platform, view to northeast.
Overtake would extend just beyond
platform to the right (south).

The key to reliable overtaking on a multiple-track railroad is to ensure that the average speed of the slower train being overtaken is sufficiently slower than the average speed of the faster overtaking train.  One of the ways of ensuring a good speed differential is to have the slower train make station stops that the faster train doesn't; each station stop is worth about 2.5 minutes.  Therefore, locating stations on the four-track overtake section is helpful.

This brings us to a lemon of a station immediately south of the midline overtake: Atherton.  Located in an area of very sparse population and jobs density, Atherton should be permanently closed.  This closure would come not only as a show of appreciation commensurate with the town's support of Caltrain modernization, but especially because census data shows clearly that Atherton is precisely where you would never place a train station.

To replace Atherton, a new Fair Oaks station should be built just 0.6 miles to the north, at the 5th Avenue grade separation.  The overtake section would be extended a bit southwards, just beyond the station, enabling locals to be passed while stopped at the central island platform that can be accessed from either side of 5th Avenue.  The new Fair Oaks stop would be equidistant from Redwood City and Menlo Park, and located in an area with very high population density that could support thriving ridership, in contrast to Atherton.

Longer Term: a Seamless Dumbarton Connection

Dumbarton rail has been an uncertain prospect for decades, with some political backing but insufficient funding.  While it may take another few decades for the money and the will to finally materialize, large concentrations of employment and the need for additional transbay corridor capacity make some form of passenger rail service inevitable.  The Dumbarton corridor also happens to be ideally suited for high-speed rail.

The key node is Dumbarton Junction, which should be reconfigured in such a way that trains can enter and leave the peninsula rail corridor swiftly and seamlessly.  This will likely involve a flyover track, enabling southbound trains to enter the Dumbarton corridor without crossing (and therefore blocking) any of the northbound tracks.  To minimize the altitude of the flyover, the Rte 84 / Woodside Road overpass would be turned into an underpass.  As for the Redwood City station, the fit would be quite tight with a 90-foot corridor width where the flyover track begins.

While the flyover may seem like an expensive solution to a problem we don't yet have, planning for it now (if not actually building it) will save money in the long run when passenger rail service grows.

Design Values

No matter how the midline overtake is ultimately configured, it must reflect design values that are clearly articulated.  One of these values should be compatibility between Caltrain and HSR.  It's not enough to talk about the "blended system" without actually taking the steps to make the two systems seamlessly interoperable, allowing any train to use any track to serve any platform.  This means no tracks can be dedicated to one operator at the exclusion of another.  Everything must be shared, including the platforms at the new mid-peninsula station.  This sharing contributes to another important value, robustness to service disruptions.  The fast-slow-slow-fast track layout is the key to ensuring that a commuter train delayed in Belmont won't create a statewide domino effect that eventually makes a train late in Los Angeles.  A third important value is future-proofing.  Infrastructure like the midline overtake will define what is possible (and not) for generations to come.  It would be short-sighted not to plan for a fast and seamless connection to the Dumbarton corridor, even if its future use isn't well-defined today.

The midline overtake is the key to an effective blended system.  When evaluating its design, ask yourself: is it compatible?  Is it robust?  Is it future-proof?

Census Driven Service Planning

The diagram at right shows where people live and work along the entire length of the rail corridor from San Francisco to Gilroy, as extracted from government census population and jobs data sets.  The diagram is also available as a PDF.

Implications for Rail Service Patterns

By simple observation of the features of the census population and job distributions along the peninsula corridor, it is possible to infer the desirable features of train service patterns that will maximize commute ridership.

1. Transbay has more than 100,000 jobs within a half-mile radius (more than every other station in the system combined). The concentration of jobs near San Francisco's Transbay Transit Center cannot be understated. This station absolutely must be served by each and every train, and it would be highly counter-productive to terminate any train at 4th and King.
    
2. Silicon Valley shows up in the jobs distribution as a broad hump, mostly homogeneous and stretching from Palo Alto to San Jose.  To serve this rich but diffuse commute market, all trains should make every stop in Santa Clara County.  There should never be any skip-stop service here, and the wider spacing of stops (relative to San Mateo County) will result in only minor trip time penalties.
    
3. San Mateo County has numerous stops, spaced more closely together and with middling jobs and population density.  To enable faster service to and from San Francisco and Silicon Valley, it makes the most sense in this portion of the corridor to operate skip-stop express service alongside local service.
    
4. South San Jose, while south of Silicon Valley, has a massive and untapped residential market that can serve as origin to jobs further north.  Tamien currently functions as a slow and infrequent addendum to the peninsula service, but should be sped up and extended to Blossom Hill.
    
5. Oakdale in San Francisco opens up a new residential market for Caltrain.  The distribution of people nearby is even denser than at 22nd Street.
    
6. The Gilroy extension doesn't make much sense.  There are so few jobs and people here that Caltrain (as primarily a commuter service) should not run to this area.  Serving Morgan Hill and Gilroy is best left for a long-distance operator such as Amtrak California.

Putting all this together, what would a service pattern look like that is tailored to the census distributions, to maximize commuter ridership?  First, it would not look like today's Baby Bullet, which severely under-serves numerous stops.  It would also look very different from Caltrain's latest planning fad, the "peak-period skip-stop zone express," used as the basis for all operational simulations in the blended service analyses published to date.  It would look like this, with distances drawn to scale:

• The Silicon Valley express links the major employment centers of San Francisco and Silicon Valley, highlighted in orange, providing a faster and better alternative to fleets of white luxury buses stuck in traffic on US 101.  It would run every 15 minutes.
   
• The San Mateo local serves all the minor stops throughout San Mateo County, terminating and originating across the platform from the Silicon Valley express at Redwood City.  This provides fast and penalty-free transfers between Silicon Valley and cities all along the peninsula.  The local turns back in Redwood City, minimizing crew and fleet requirements while still providing service every 30 minutes.
   
• Stops in San Francisco and San Jose that have very large residential markets are served in the peak commute direction only.

This service pattern is also well-suited to future blended HSR service: a southbound high-speed train and a closely-following Silicon Valley Express can overtake and catch up (respectively) with a San Mateo local by using a four-track mid-line overtake facility constructed from San Mateo to Redwood City.  Because the local turns back at Redwood City, it does not impede traffic in Santa Clara County.

The time has come to fundamentally rethink peninsula rail service patterns.  Caltrain's "peak-period skip-stop zone express" is almost certainly not the best solution for meeting future demand; a much wider range of options must be considered.

Formulation of a Service Quality Metric

The quantitative formulation of an overall quality metric, which can be extracted from an arbitrary timetable, is necessary to objectively answer the question “is proposed timetable A better than proposed timetable B?”

Such metrics facilitate the trade-study and optimization process of planning a new timetable, and must take into account several factors, including not just the quality of the service provided to passengers but also other factors that passengers don’t think about, such as robustness to disruption, fleet size and crew time considerations.

For today, however, we will focus exclusively on quantifying the quality of the service provided to passengers.  This particular formulation proceeds in eight reasonably simple steps, pulling together earlier information on timetable metrics and demographics.  It is only one example of how one might formulate a service quality metric, something that Caltrain has never explicitly done and could benefit greatly from doing as they share the pros and cons of various blended service plans.  This is one way to do it; what's theirs?

Step 1: Extract trip time and wait time statistics for each origin and destination pair.  By straightforward analysis of the timetable, one can figure all the possible trips between any origin station A and destination station B (including transfers) during a one-hour span during the morning peak.  One can then determine (in units of time):

1. The average trip time between A and B (Tmean_AB)
2. The fastest trip time between A and B (Tmin_AB)
3. The average wait between trips that connect A and B (Wmean_AB)
4. The longest wait between trips that connect A and B (Wmax_AB)

The first two metrics measure trip time on board the train, and the next two can be used as a proxy for measuring typical wait times on the platform.  The trip time and wait time figures are intrinsic to the timetable and can be extracted by a computer program.

Step 2: Compute an “effective” trip time from A to B by computing a weighted sum of the time components extracted above. This is where judgment calls start to be made. Taking into account the waiting times Wmean and Wmax is just as important as the actual trip times Tmin and Tmean, in order to properly account for the frequency of service. For example, the effective trip time could be defined as:

Teff_AB = (30% of Tmin_AB + 70% of Tmean_AB) + (20% of Wmean_AB + 15% of Wmax_AB)

The trip time term (30% of Tmin_AB + 70% of Tmean_AB) accounts for some trips being shortened by express service. The waiting time term (20% of Wmean_AB + 15% of Wmax_AB) properly penalizes long service gaps, but remains shorter than the waiting time incurred when the passenger shows up randomly, which is 50% of Wmean_AB.  This lower weighting reflects the fact that passengers don’t show up randomly, but usually time their arrival at origin A for a particular trip to destination B.  For example, when trips are available every 15 minutes, the waiting term works out to a quite reasonable 5 minutes. The effective trip time is a reasonably good measure of how long it will take you to get from A to B.

Step 3: Determine the “effective” speed between origin A and origin B. This is simply distance divided by time, or: V_AB = d_AB / Teff_AB where d_AB is the distance between A and B. This process is repeated for every origin and destination pair A-B, and describes not the speed of a train, but the average speed of a typical trip from A to B including waiting time, based only on the available service provided by the specific timetable being considered.

Step 4: Compute weighting by population and jobs.  This is where census data enters the calculation, as it must.  For the morning rush hour, since ridership consists primarily of people going from their home near A to their work near B, we calculate a potential ridership weight based on how many people live near A and how many people work near B.  This simply reflects that if a lot of people live near A and work near B, it is more important to provide fast service between A and B than between other station pairs where fewer people and jobs are located.

The “home weight” Whome_A of origin station A is a simple gravity sum (1/r squared law) of the residential population, taken from the 2010 census, as described previously in greater detail.  Each person is divided by the square of how far they live from station A, to reflect that people who live further away from the station are less likely to use it. To prevent over-counting people who live very close to the station (where the 1/r squared term diverges), anyone living closer than ¼ mile from the station is considered to live ¼ mile away.  The resulting weights are shown at left, in orange.

Similarly, the “work weight” Wwork_B of destination station B is a simple gravity sum of the number of jobs over $40k, again taken from census data. Each job is divided by the square of how far it is from station B, to reflect that people who work further away from the station are less likely to use it. Once again, to prevent over-counting jobs located very close to the station, any job closer than ¼ mile from the station is considered ¼ mile away.  The resulting weights are shown at right, in blue.

Step 5: Compute weighting by distance. Regardless of where people live and work, there are upper and lower limits to how far they will typically commute by rail. Extremely short trips are less likely because of the overhead of access and egress to and from the station at each end of the journey. Conversely, extremely long trips are less likely because of their sheer duration.  As it turns out, the typical rush hour trip on Caltrain turns out to be about 25 miles, or 40 km.

For our purposes, the distance weighting is constructed by drawing a curve with a peak at 40 km. This distance weight starts off at zero for a trip distance of less than 7 km (reflecting no demand for such short trips), peaks at a distance of 40 km, and decays slowly thereafter.  Converted to miles, it looks like the figure at left.  The underlying math to draw this curve is a Rayleigh distribution with a peak at (d-7) = 33, where d is the trip distance in km.

Step 6: Combine the population, jobs and distance weights to obtain a ridership potential matrix.  The ridership potential matrix R is a matrix of size N squared, where N is the number of stations.  Each element R_AB of this matrix represents the "potential" ridership (in arbitrary relative units) that can be tapped into during the morning commute from origin A to destination B.  This ridership potential matrix has an important property: it is independent of any timetable, and concisely describes the underlying demand that inherently exists out there--regardless of how or whether that demand is met by rail service.  Each element R_AB is given by the product:

R_AB = Whome_A * Wwork_B * Wdistance_AB

Note that the matrix R is not symmetric, because the number of residents and jobs near each station differs.  For example, far more people will want to commute to SF Transbay in the morning than from it, since the number of jobs within a half mile of that station is greater than all the jobs within a half mile of every other Caltrain station all the way to Gilroy combined.

Step 7: Compute the service quality matrix. The service quality matrix Q is again a matrix of size N squared, where N is the number of stations. Each element Q_AB of this matrix represents the quality of morning rush hour service from station A to station B, and is given by the following formula:

Q_AB = R_AB * V_AB

This combines R_AB, the timetable-independent ridership potential from origin A to destination B, with V_AB, the timetable-dependent effective speed from A to B.  If you have a preferred AM origin and destination (as most commuters do), then you can compare your Q_AB for various timetables to see how any given timetable will meet your own specific needs.

Step 8: Extract overall service quality scores. The service quality metrics must be bench marked against some reference, so they are simply normalized against the most current timetable.  That means today's timetable will score 100, by definition.  By adding the elements of Q over all possible origin and destination pairs, we can quantify the degree of service improvement and compute a score for the entire timetable as well as a score for each individual station. The overall timetable service quality score is S = ΣQ / Sref, i.e. the sum of all the elements of Q divided by the corresponding sum for today's timetable.

An entire timetable can now be distilled to its essence, a single service quality score.

We are now empowered to compare various timetables and understand quantitatively the pros and cons of each.  This method will tell you objectively whether timetable A provides better overall service than timetable B--and if you happened to disagree with the scoring outcome, then your argument would be with the scoring method and not any particular detail of this or that proposed timetable.  Beyond the mathematical minutiae of the rather simple scoring method presented here, the larger point is that there needs to be a defined scoring process and a framework for stakeholders to discuss what makes a good timetable.  This scoring process is absolutely essential for planning future blended service on the peninsula.  Caltrain's approach so far has been to prescribe a certain skip-stop pattern (see Tables 7 and 8) and restrict all analysis to that particular pattern, seemingly without regard to overall service quality!

Is Demand-Based Planning a Myth?

Original photo by qviri

With over 80% of riders using Caltrain to commute to their jobs during rush hour, one would think that the service would be planned around where people live and where people work, using cold hard numbers from the census.  That's known as traditional demand-based planning: provide service where the most people will use it.  It's not rocket science, and demand-based planning is used all around the world to plan excellent rail service.

But not here on the peninsula.

In a contrarian argument made circa 2005, Caltrain's operations staff claimed that demand-based planning is a myth. (14 Mb PDF file)   At the time, Caltrain was crowing to its industry peers about the success of the Baby Bullet.  The keys to success included "Questioning Traditional Planning Processes" and "Trusting Your Intuition".  Numbers don't matter, just go with your gut!

In the years since, there has been plenty of hard evidence that the Baby Bullet has severely reduced ridership at many locations, especially in Santa Clara County.  Indeed, data from the 2010 census can be correlated to the latest Caltrain ridership data without ever looking at a timetable to reflect quite accurately which Caltrain stops are under-served and falling short of their ridership potential.

Maybe demand-based planning isn't such a myth after all.  Maybe numbers don't lie.  Here's hoping that objective, quantitative metrics will play a central role in planning future blended operation scenarios with high-speed rail.  This stuff is too important to trust anybody's intuition.

Peninsula Rail Corridor Census

The U.S. Census Bureau provides an astonishing array of fine-grained statistics on population and jobs along the peninsula rail corridor.  When thinking about the future of peninsula rail service, and especially in deciding quantitatively how good a proposed timetable might be, or where stations should be placed, or how HSR should mesh with Caltrain in a 'blended' scenario, the basic consideration should be where people live and work.

Annual ridership counts provide one way of planning your timetable: simply add more service to the stops that get a lot of ridership.  This becomes a self-fulfilling prophecy with ridership patterns becoming distorted by the timetable, as observed with the Baby Bullet Effect.  Teasing apart the timetable-induced distortion from the underlying (and often untapped) ridership demand is impossible, so it is necessary to go back to the raw population and jobs data to build the full picture.  That is where the census really delivers.

Where People Live

Figure 1

The 2010 census provides the most recent snapshot of the population distribution on the peninsula, on a block-by-block basis that includes over 45,000 locations in the three Caltrain counties.  By tallying up how many people live within 1/4, 1/2, 1 and 2 miles of each Caltrain station location, you can build Figure 1.  This chart reveals where the population is densely concentrated around stations (e.g. San Mateo), or sprawled out (e.g. Sunnyvale).

Observations on the population numbers:

• The new Oakdale station long proposed by San Francisco (with little support from Caltrain) could tap into more residential population than just about any other stop along the peninsula, or even 22nd Street.
• The population density doesn't suddenly drop off at the southern end of the Caltrain-owned right of way in San Jose, where service suddenly drops off.  There are large concentrations of under-served population within a mile of the Tamien and Capitol stops, accounting for more than 3 times as many people as live within a mile of the San Jose Diridon station.
• A stop like Broadway (Burlingame) with zero weekday rail service has more people living near it than Millbrae, site of the all-important BART intermodal station.  Other stations with poor Caltrain service (San Antonio, Cal Ave, San Bruno, Burlingame, Belmont, Santa Clara) have more people living nearby than stops with the best service, such as Palo Alto.

Figure 2

To assign to each station location a single weighting factor that quantifies that station's accessibility for nearby residents, regardless of distance, one can sum up each person divided by the square of how far away they live.  This inverse-square relationship is empirical, but captures the fact that people who live far away from a station are less likely to use it; its use in ridership modeling is not unprecedented.  A 1/r law would fall off too slowly, with the same number of people using the station from 1/2 mile away as 2 miles away (assuming constant population density).  A 1/r cubed law would fall off too quickly, with only 1/16th as many riders from 2 miles away as from 1/2 mile away.  As it turns out, the precise value of the exponent--if not exactly two--doesn't really drive the relative weights that strongly.  Only one small tweak has been applied to prevent people who live very close to a station from skewing the results: anyone living closer than 1/4 mile is considered 1/4 mile away.  The resulting inverse-square population weights for each station location are shown in Figure 2.

Where People Work

Figure 3

The Census Bureau publishes extensive statistics on local employment dynamics, providing block-by-block data on the number and distribution of jobs, pay levels, and industries.  The latest data set as of this writing is from 2009 (based on geographical data from the 2000 census covering over 32,000 locations in the three Caltrain counties).  The analysis presented here is based on raw data files, but the data can also be analyzed interactively using the Census Bureau's On The Map application.   Figure 3 shows how many jobs are located within 1/4, 1/2, 1 and 2 miles of each Caltrain station location.  Only the jobs worth more than $40k a year are shown, since lower-income jobs are less likely to require commuting (only about 15% of Caltrain riders earn less than $40k, and the average household income of a weekday peak Caltrain rider is over $100k).

Observations on the jobs numbers:

• Not so surprisingly, there is a concentration of jobs in the vicinity of the future Transbay Transit Center, adjacent to the financial district.  What is more surprising is just how massive that concentration is: Transbay has more jobs within a half-mile radius (over 100,000) than all the other Caltrain stations combined, from 4th & King all the way down to Gilroy!
• Job sprawl shows up in Santa Clara and southern Palo Alto (and most of Silicon Valley, really) in the form of few jobs near stations but many jobs within a mile or two.  Mountain View, despite its status as a major Baby Bullet stop, and home of Google, is not a particularly large job center.

Figure 4

Again, assigning to each station a weighting factor that quantifies that station's accessibility to nearby jobs, we apply the same inverse square relationship to obtain the job weights for each station location shown in Figure 4.  Note that Transbay goes way off the chart.

The Ridership Potential Matrix

Since 86% of riders during the weekday peak are commuters, the distribution of population and jobs can be used to construct a relative weight for the ridership that could potentially be generated between any given origin and destination (O&D) pair.  This is the ridership potential matrix.  The eventual purpose of this matrix is to help derive a single figure of merit for timetables, on an apples-to-apples basis, for how much of the potential ridership is tapped based on the service metrics for each O&D pair.  When considering any given timetable, this weighting scheme ensures that O&D pairs that have a lot of population and jobs at each end (such as 4th & King and Palo Alto) are given more importance compared to O&D pairs with lower population and fewer jobs (such as Atherton and Bayshore).

It is important to note that this ridership potential matrix is completely independent of how each O&D pair is connected by rail service; it holds true for any timetable.  It is solely a product of census data and the geographic location of each station.  A timetable must then be designed to unlock the maximum potential ridership.

The ridership potential matrix works like this: take for example station 1 and station 2, with respective population and job weights P1, P2, J1 and J2.  The weight for morning peak trips from origin 1 to destination 2 is P1*J2 (for people living near station 1 and working near station 2).  Conversely, the weight for morning peak trips from origin 2 to destination 1 is P2*J1 (for people living near station 2 and working near station 1).  When you multiply all the population weights from Figure 2 by all the job weights from Figure 4, you get a basic ridership potential matrix.  But there's a bit more to it than just people and jobs.

Distance Considerations

Regardless of where people live and work, there are upper and lower limits to how far they will typically commute by rail.  Extremely short trips are less likely because of the overhead of access and egress to and from the station, at each end of the journey.  Conversely, extremely long trips are less likely because of their sheer duration; regional commute patterns are not just a factor of train service considered in isolation, but also driving times.  That's why we will make the assumption that the distance distribution of commutes, generally speaking, is independent of the quality of train service--and that no foreseeable rail service pattern could significantly alter it.  Good service might lead to greater market share for rail, but the underlying distance distribution will be assumed not to budge.  This allows us to apply a (timetable-independent) distance distribution to the ridership potential matrix.

Figure 5

Caltrain ridership surveys show that the average trip length on the peninsula rail corridor during the weekday peak is about 25 miles.  The distance weighting function will be modeled as a Rayleigh distribution with a value of 0 at 0 miles and a peak of 1 at 25 miles-- for no particular statistical reason other than it ends up looking about right, as shown in Figure 5.

Each element of the ridership potential matrix is now the product of three factors: the distance weight based on the distance between origin and destination; the population weight at the origin station; and the job weight at the destination station.  This simple formulation yields the morning peak values shown in Figure 6 as a bubble graph (numerical values are available as a tab-delimited text file).  The evening peak is described by the transpose of the matrix, i.e. origin and destination switch places.  The distance-weighted ridership potential matrix is now ready for use in the quantitative analysis of past, present and future timetables, a topic that will be covered in upcoming posts revisiting the topic of service metrics.

Figure 6

Figure 8

Figure 7

In the meantime, we can explore other interesting aspects of the ridership potential matrix.  For example, summing the nth row together with the nth column of the matrix allows us to build a single weighting factor for the potential ridership at each stop including both the morning and evening peaks, i.e. a measure of the ridership distribution that could exist if it were tapped with excellent service, shown in Figure 7.  These weights can then be compared to the actual Caltrain ridership realized in 2011, yielding the scatter plot in Figure 8.  This comparison provides another more fundamental way (much better than historical ridership patterns) to visualize which groupings of Caltrain stops are under-served, and is amazingly accurate considering that it was constructed without ever looking at a timetable.

Key conclusions:

• Access to Transbay would provide a step-change improvement in Caltrain service, with probable ridership gains of more than 25%.  Terminating any weekday peak train at 4th & King, as is inexplicably planned by Caltrain, is a huge mistake.  Agency turf battles with BART and the CHSRA regarding whether or how to pay for the downtown extension tunnel, and how to share platforms at Transbay, must be fought and won.
• Underlying ridership demand is not accurately reflected by realized ridership, which suffers from severe timetable distortion.  Future service planning, and in particular the timetables assumed for the ongoing 'blended' operations analysis, must be based less on realized ridership and more on fresh census data--even if not using the simplified approach described here.
• For the same reason that every Caltrain should serve Transbay (the huge concentration of jobs in San Francisco), HSR service that does not provide a one-seat ride into Transbay is a non-starter.