appj

APPENDIX J - ENHANCEMENT Evaluations

Ranging Structure Overview

CDTC developed a basis for evaluating candidates for funding under the Transportation Enhancements Program (TEP). Under this approach, candidates would be scored on eight criteria set forth in the TEP Guidebook:

Benefit to Enhancement Region and Environment

Enhancement of Transportation Plans and Projects

Relationship to/Support for Other Plans and Projects

Size of Matching Share and Assurance of Availability

Direct User Benefits to Immediate Proposal Area and Environment

Innovation, Creativity and Mix of Activities

Supportiveness of Master Planning in Recognized Areas of Special Significance

Level of Community and Regional Support

The first two of these criteria would be worth 20 points each and the remainder worth 10 points each, for a total of 100 points. For the two 20-point criteria, ten points each would be applied to each of two subcriteria identified in the ranging structure below. In addition, two of the ten-point criteria -- "Innovation, Creativity and Mix of Activities" and "Level of Community and Regional Support" -- are broken down into two five-point subcriteria based on the Guidebook's indicating some significantly different elements to consider for either. For the "Innovation…" criterion, innovation and creativity are significantly different from the mix of activities. Thus, proposals could get up to five points for the two former elements and up to five points for the latter. For "Level of Community and Regional Support", extensive efforts to reduce project costs and letters of support are cited as determining factors; each of these elements would also be worth up to five points.

The ranging structure gives sample indications of how points are awarded for individual criteria, and allows the evaluator to review guiding language from the TEP Guidebook and CDTC interpretations and key in scores accordingly. The structure is reflected in a spreadsheet template, a completed version of which will be printed out and kept on file for each proposal. In addition, narrative rationales for scores may be included in the printouts.

Structure Basis

For four criteria, the point scores are proposed to be direct functions of some quantifiable attribute:

Relationship to/Support for Other Plans and Projects: Points are assigned based on how many such plans or projects are cited and/or known (and can validly be considered to be supported by the proposal)

Size of Matching Share/Assurance of Availability: Points are assigned based on where the indicated matching share falls into a series of percentage ranges

Project is Innovative: Points for this subcriterion of "Innovation, Creativity and Mix of Activities" are assigned based on definable unique features and "model project" potential; point assignments under the "mix" subcriterion would be based on how many TEP-eligible activities were incorporated into the proposal

Supportive of Master Planning in Recognized Areas of Special Significance: Points are assigned based on how many appropriate areawide or statewide plans are cited and/or known (and, again, can validly be claimed)

For the remaining criteria, the proposed ranging structure sets forth four illustrative "levels of success" that a proposal might achieve, with narrative descriptions of each: zero percent (no success); 20 percent (low success), 50 percent (medium success), or 100 percent (high success). These levels would correspond to zero, two, five and ten point scores for the ten point criteria, and zero, one, 2 1/2 and five points for the five point subcriteria. Again, these are illustrative; evaluators would award whatever point scores within the maximums were deemed appropriate based on individual proposal attributes.

In examining the ranging structure, it should be borne in mind that by such measures as transportation benefit or economic development, most TEP proposals would be seen at best as only having "low success" (that is, two points out of ten) potential compared to activities such as highway construction or the development of a new office building. It is arguably not appropriate to consider the potential benefits of Enhancement-type projects against the reference of all possible investments. Thus, it is proposed that the maximum potential (100%) "level of success" be based on what is possible for TEP-type projects in the Capital District. This determination will require a combination of staff knowledge of existing TEP-type projects (including completed projects which were not funded under TEP but would have been eligible) and what the theoretical "best case" benefit of an Enhancement project could be. Staff would document the bases for all point scorings, and would have this supporting information available if needed when it presents the results of its reviews to the Planning Committee for approval before transmission to the TEAC.

Test Application of Evaluation Structure

The evaluation structure was applied to five ISTEA-era TEP proposals: the Albany County/Schenectady County Helderberg-Hudson Rail Trail; the Colonie Pedestrian/Bicycle Circulation System; the acquisition of the Taconic Ridge Tarr Parcel in Rensselaer County; the Clifton Park Farmer's Bridge; and the Albany County Shaker Barn project. The aim was to see how the proposed tool would score projects previously judged by CDTC as strong or weak based on the checklist basis approved by the Planning Committee in 1993. (The same CDTC staff member who performed the checklist-based evaluations of these proposals applied the new evaluation structure.) A key concern driving the test was that of whether the tool would fairly evaluate the full range of TEP-eligible activities. Both successful and unsuccessful TEP proposals from the past were considered, although it should be noted that one of the concerns raised regarding the ISTEA-era TEP was that there was not a consistent relationship between MPO endorsement and success in winning TEP funding. It is expected that this relationship will be stronger in the TEA-21 era.

The results were as follows:

Criterion	Max Points	H-H Rail Trail	Colonie Ped/ Bike System	Taconic Ridge Parcel	Farmer's Bridge	Shaker Barn
1. Benefit to Enhancement Region and Environment	20	10	6	10	4	4
2. Enhancement of Transportation Plans, Projects	20	14	16	2	4	4
3. Relationship to/Support for Other Plans, Projects	10	8	8	6	2	0
4. Size of Matching Share, Assurance of Availability	10	6	2	2	2	2
5. Benefits to Immediate Proposal Area and Environment	10	7	9	8	5	8
6. Innovation, Creativity, Mix	10	7	0	3	1	2
7. Supportive of Master Planning in Recognized Areas of Special Significance	10	8	0	8	6	6
8. Level of Community, Regional Support	10	9	4	3	2	5
TOTAL	100	69	45	42	26	31
"Yes" responses to CDTC TEP Evaluation Checklist	7	5	7	1	2	2
TEP Funded?		YES	NO	YES	NO	YES

The sample reviews suggest that while the Helderberg-Hudson Trail fared particularly well, the variety of criteria seems to ensure that no project will be predestined to score extraordinarily well simply by virtue of what type of project (e.g., a trail) it is. High scores would be functions of a proposal's ability to address several concerns. As this analytic basis considers a broader range of attributes than the checklist basis did, transportation function is important but not a dominant influence of the aggregate scores.

To the extent that the Taconic Ridge parcel purchase can be taken as illustrative, the proposed evaluation method does appear to provide better opportunities for projects other than transportation facilities than the checklist did. As for the Shaker Barn proposal, its historic preservation and tourism elements manifested themselves in a strong score for Criterion 5. Beyond this measure, however, it could be argued that the project does not fare well because while desirable, there was nothing truly distinguishing the project by any of the other criteria.

Ranging Structure Details

Summary Sheet

Project Name:
Project Sponsor:
Score:
		Rank:
Reviewer:

			This
		Max	Proj.	Summary of Criteria
	Level	Score	Score
	A	10	0	1a. Benefit to Enhancement Region and Environment: Environmental
	A	10	0	1b. Benefit to Enhancement Region and Environment: Economic
	A	10	0	2a. Enhancement of Transportation Plans and Projects: Access/Patronage
	A	10	0	2b. Enhancement of Transportation Plans and Projects: Transportation System
	A	10	0	Relationship to/Support for Other Plans and Projects
	A	10	0	Size of Matching Share, Assurance of Availability:
	A	10	0	Direct User Benefits to Immediate Proposal Area and Environment:
	B	5	0	6a. Innovation, Creativity, and Mix of Activities: Project is innovative or could serve as a model for similar enhancement projects.
	B	5	0	6b. Innovation, Creativity, and Mix of Activities: Project encompasses two or more eligible transportation enhancement activities.
	A	10	0	Supportive of Master Planning in Recognized Areas of Special Significance
	B	5	0	8a. Level of Community and Regional Support: Special Efforts to Reduce Costs
	B	5	0	8b. Level of Community and Regional Support: Letters of Support, Resolutions, Endorsements

	TOT	100	0

Detailed Discussions of Criteria and Scores
	Level		This
	Of		Proj
Level	Success	Score	Score	Criterion 1a:
A			0	Benefit to Enhancement Region and Environment: Environmental

	0%	0.0		NONE (project not likely to produce ANY environmental benefit)
	20%	2.0		LOW (project likely to have some minimal environmental benefit, e.g., minor improvements to an historic site)
	50%	5.0		MEDIUM (project likely to have modest but significant environmental benefit, e.g., by preserving an historic structure)
	100%	10.0		HIGH (project likely to provide substantial environmental benefit, e.g., by encouraging significant shifts from driving to cycling or walking)

				Criterion Definition: The extent to which the project would preserve or positively influence natural or cultural resources, scenic quality, air or water quality, wildlife habitat or migration.

				In general, this category focuses on the "natural" environment; however, the Guidebook notes that the rater has broad discretion to interpret and define these terms. For example, the rater will form an opinion on "what is a cultural resource?" (there are many answers: one interpretation might be "areas of historical or archeological significance", while another could be "areas where human social interactions may occur"). Examples of indicators might be:

				Natural Resources conserved or protected
				Cultural Resources conserved or protected
				Scenic quality preserved or enhanced
				Air quality directly improved as a result
				Water quality directly improved as a result
				Wildlife habitat/migration areas are preserved, restored, created, or otherwise enhanced


Notes to Score Selection
___________________________________________________________________________________
___________________________________________________________________________________
___________________________________________________________________________________
___________________________________________________________________________________
___________________________________________________________________________________

Note: For each criterion, space will be provided as in the example above to enter information to clarify the basis for assigning a particular score. To save space, this is not represented in the remaining criterion discussions.

Also note that the examples provided for low/medium/high potentials under each criterion should not be taken to be the only examples with regard to benefiting groups or project types which could qualify for points at the indicated level of success. Furthermore, the "medium" and "high" determinations could be the results of some cumulative consideration, e.g., the achievement of more than one type of "low-level" benefit.


	Level		This
	Of		Proj
Level	Success	Score	Score	Criterion 1b:
A			0	Benefit to Enhancement Region and Environment: Economic

	0%	0.0		NONE (project not likely to produce ANY economic benefit)
	20%	2.0		LOW (project likely to have some minimal economic benefit, e.g., a bike trail that could attract some day-trippers)
	50%	5.0		MEDIUM (project likely to have modest but significant economic benefit, e.g., is supported by a market study saying that the activity it fosters can attract tourists)
	100%	10.0		HIGH (project likely to provide substantial economic benefit, e.g., is necessary companion to creation of a staffed visitors’ center)

				Criterion Definition: The extent to which the project would improve the quality of life through job creation, increased tourism, economic development, balanced distribution of funds and other socioeconomic factors.

				This category focuses on the potential for positive economic impacts resulting from an enhancement project. Examples follow:

				Additional jobs are created in the community
				Existing jobs will be retained within the community
				Tourism and visitor revenues will be enhanced through:
				- Additional hotel occupancy, increased restaurant and retail sales
				- Increased potential for "return trips"
				Economic Development potential (e.g., marketability of the community) is enhanced through:
				- Improved community aesthetics
				- Perception of a higher "quality of life"
				Economically challenged individuals are assisted.


	Level		This
	Of		Proj
Level	Success	Score	Score	Criterion 2a:
A			0	Enhancement of Transportation Plans, Projects: Access/Patronage

	0%	0.0		NONE (project not likely to produce ANY transportation access/patronage benefit)
	20%	2.0		LOW (project likely to have some minimal transportation access/patronage benefit, e.g., would provide a better scenic view from a trail)
	50%	5.0		MEDIUM (project likely to have modest but significant transportation access/patronage benefit, e.g., provides a new bicycle or pedestrian connection)
	100%	10.0		HIGH (project likely to provide substantial transportation access/patronage benefit, e.g., remedies a major barrier/hazard to pedestrian travel in a dense urban area)

				Criterion Definition: The extent to which the project would increase or improve access to activity centers (business, school, recreation, shopping, etc.).

				The focus of this category should be on the enhanced mobility (especially with non-traditional modes) of people or on significant improvement in the quality of the trip experience (e.g., improved safety, access to historical or scenic sites, etc.). Examples here are best expressed in the form of questions:

				How many people will use these new connections?
				What is the current level of connectivity/access (i.e. how dramatic are effects of the proposed improvements)?
				Is user safety/security a current issue?
				Is access guaranteed to all individuals (e.g., ADA, private ownership are examples of issues)?
				How will this project enhance the "trip experience"? (e.g., billboard removal, preservation or establishment of scenic overlooks, historical markers, etc.)
				What activity centers will be connected?


	Level		This
	Of		Proj
Level	Success	Score	Score	Criterion 2b:
A			0	Enhancement of Transportation Plans, Projects: Transportation System

	0%	0.0		NONE (project not likely to produce ANY transportation system enhancement)
	20%	2.0		LOW (project likely to have some minimal transportation system enhancement, e.g., will allow bike commuters to lock their bicycles at bus stops)
	50%	5.0		MEDIUM (project likely to have modest but significant transportation system enhancement, e.g., will provide a pedestrian refuge or signal actuation button at a busy intersection)
	100%	10.0		HIGH (project likely to provide substantial transportation system enhancement, e.g., completes a safe pedestrian travel route in a congested area)

				Criterion Definition: The extent to which the project would reinforce or complement the regional transportation system, fill deficiencies in the system, have multi-modal aspects, or connect transport modes.

				This category concentrates on the development of the intermodal transportation system envisioned by the ISTEA legislation and reinforced in TEA-21. Whereas the previous category looked at how the proposed project meets user "demand", this category looks at the "supply" aspects of the transportation equation. Examples include:

				Transportation modes being connected (e.g., bikes and pedestrians, bikes and buses, bikes and autos, trains and pedestrians, etc.). Also, projects identified in transportation plans; a part of continuing or ongoing transportation programs.
				System deficiencies being addressed (e.g., pedestrian circulation systems, bikeway systems, etc.).


		This
		Proj
Level	Score	Score	Criterion 3:
A		0	Relationship To/Support for Other Plans, Projects:

	0.0		NONE (project not likely to further any local, regional or state plan goals)
	2.0		LOW (project likely to further goals in one cited or known plan)
	5.0		MEDIUM (project likely to further goals in two or three cited or known plans)
	10.0		HIGH (project likely to further goals in more than three cited or known plans)

			Criterion Definition: The extent to which the project would implement goals in regional plans or other federal, state or local plans.

			The linkage to existing plans is critical. This is particularly true for projects within urbanized areas under the jurisdiction of a Metropolitan Planning Organization (MPO). Under the law, MPO's must not only approve projects for programming in their Transportation Improvement Program (TIP), the projects must relate to a long-range plan. Remember that a formal action from a MPO (e.g., resolution) represents the full support and approval of all of the member governments and participants in the metropolitan region. If a project is known to be consistent with, or actually may implement some aspect of various plans, ordinances, local master plans, etc., it is appropriate to make note of that fact.


		This
		Proj
Level	Score	Score	Criterion 4:
A		0	Size of Matching Share, Assurance of Availability:

	0.0		<20% match (ineligible)
	2.0		20-24.99% match
	6.0		25-34.99% match
	10.0		35% or greater match

			Criterion Rationale: A 20% minimum match is required; the provision of a match in excess of 20% benefits the overall program as it allows federal funds to be used for additional enhancement projects. Also, given the fiscal difficulties which can exist in providing more than the minimum required match, "overmatch" is a noteworthy indication of a sponsor's/applicant's commitment to the proposal.


	Level		This
	Of		Proj
Level	Success	Score	Score	Criterion 5:
A			0	Direct User Benefits to Immediate Proposal Area and Environment

	0%	0.0		NONE (project not likely to produce ANY direct user benefits of these sorts)
	20%	2.0		LOW (project likely to have some minimal direct user benefits of these sorts, e.g., primarily benefits a small number of "through trips")
	50%	5.0		MEDIUM (project likely to have modest but significant direct user benefits of these sorts, e.g., expected to get a great deal of use by people living nearby)
	100%	10.0		HIGH (project likely to provide substantial direct user benefits of these sorts, e.g., will enable disabled people to use a facility they previously could not)

				Criterion Definition: The extent to which the project would increase availability, awareness or protection of historic, community, visual or natural resources. Also, the extent to which it identifies groups in the population, including people with disabilities, who will benefit from or are likely to use the project. The variety of user groups and the number of users will be considered, as will the preservation or enhancement of related unique features.

				There is some similarity between criterion 1a. and this one; the distinction lies in this criterion's focusing on the direct user benefits of the proposed project. "Direct" benefits can be thought of as "local" impact, compared to criterion 1a's regional impact. Illustrations follow:

				Number of people/groups of people who will benefit (e.g., pedestrians, cyclists, equestrians, skiers, travelers, etc.).
				Preserves/enhances historic resources (e.g., historic transportation facilities, structures, etc.).
				Preserves community resources (e.g., neighborhoods, cultural facilities, gathering areas, etc.).
				Provides accessibility to people with disabilities.

			This
			Proj
Level		Score	Score	Criterion 6a:
B			0	Innovation/Creativity/Mix: Project is innovative or could serve as a model for similar enhancement projects.

		0.0		Project is routinely organized, designed, planned
		1.0		Project has a couple of unique characteristics
		3.0		Project has unique characteristics / some model potential
		5.0		Project is extremely unique / definitely a model

				Criterion Definition: The level of "innovativeness" or the suitability of the project as a "model" for other projects. Unique design or application, new technologies, development of public/private partnerships and multi-jurisdictional projects, are all good examples.


		This
		Proj
Level	Score	Score	Criterion 6b:
B		0	Innovation/Creativity/Mix: Project encompasses two or more eligible transportation enhancement activities.

	3.0		2-3 eligible activities
	5.0		4+ eligible activities

			Many transportation enhancement project proposals may technically encompass two or more eligible activities. If they do, the TEAC will consider this fact in their rating. However, each individual aspect of the proposal should "stand alone" in the sense; if the project were split by category, each would qualify on its own merits: (e.g., landscaping might be only a side-effect to the development of a scenic overlook and probably would not receive extra credit).


		This
		Proj
Level	Score	Score	Criterion 7:
A		0	Supportive of Master Planning in Recognized Areas of Special Significance

	0.0		No connections to broader plans cited or discernible
	6.0		Project furthers the goals of one areawide or statewide plan
	8.0		Project furthers the goals of two areawide and/or statewide plans
	10.0		Project furthers the goals of three or more areawide and/or statewide plans

			Criterion Definition: The extent to which the project would further the goals of current plans of statewide or broad area special significance, such as the Adirondack and Catskill Parks, the Hudson River Valley Greenway, Coastal Zones, Urban Cultural Parks, State Open Space Conservation Plan, the Statewide Comprehensive Outdoor Recreation Plan, and the Canalway Plan.


	Level		This
	Of		Proj
Level	Success	Score	Score	Criterion 8a:
B			0	Level of Community, Regional Support: Special Efforts to Reduce Costs

	0%	0.0		NO special effort
	20%	1.0		LOW (some) special effort (e.g., part of the site work for a project to be done by community group)
	50%	2.5		MEDIUM special effort
	100%	5.0		HIGH (outstanding) special effort (e.g., substantial use of other grants to complement this project/compound its benefits)

				Criterion Definition: The extent to which efforts have been made to reduce project costs (e.g., through the use of volunteer labor and other goods and services), and other efforts to improve the cost-effectiveness of the project (relationship between performance or productivity and the annualized total project cost). Increasing the match does not reduce the project cost. This criterion should not be confused with criterion 4.

				While eligible as match funds, the donation of goods and labor, particularly from "grass-roots" organizations, for the completion and maintenance of the project deserve special attention if proposed to be non-participating or truly "donated" to the project. In addition, other efforts, such as financial packaging or the use of other grant funds, which reduce the overall cost of the eligible project also deserve merit.


	Level		This
	Of		Proj
Level	Success	Score	Score	Criterion 8b:
B			0	Level of Community, Regional Support: Letters of Support, Resolutions, Endorsements

	0%	0.0		NO support
	20%	1.0		SOME (LOW) support (e.g., an official letter of support or two)
	50%	2.5		MEDIUM support (e.g., a few letters, from both officials and community groups)
	80%	4.0		HIGH support (e.g., support from officials, community groups in greater number than "medium")
	100%	5.0		HIGHEST (outstanding) support (e.g., several letters from officials, community groups, advocacy groups, elected representatives, newspaper editorial articles in support)

				Criterion Definition: Letter(s) of support from elected officials; endorsement action from local governments (resolutions, etc.); endorsement action from local governments (resolutions, etc.); letters of support/endorsement actions from interest groups (e.g., Chambers of Commerce, advocacy groups, neighborhood associations, etc.) "Highest" support would reflect the inclusion of a substantial number of letters of support or other references to endorsement.

				This is a critical category in that it represents the level of community and political support for the project. While transportation projects are often delayed (or terminated) as a result of significant opposition, projects that have the full backing of community groups and leaders/elected officials have a higher completion rate. Projects that demonstrate evidence of a combination of both "grass roots" support and support from the appropriate officials are more favorable than those that do not. The degree of support is also critical: letters from individuals are good, but resolutions, petitions, or other formal actions of support by groups of people are better.