Open Data Policy
Enhancing Open Government and Transparency
DataSF.org access to San Francisco's datasets
A city that drafts an open data policy and improves access to machine readable data will enhance government transparency, accountability, public participation and collaboration. Increasing access to government datasets encourages a city’s citizenry, business community, and visitors to create innovative civic tools and applications at no additional cost to City government.
A city that drafts an open data policy and improves access to machine readable data will enhance government transparency, accountability, public participation and collaboration.
A two pronged approach has been established by the City and County of San Francisco. DataSF.org is the public presence, a clearinghouse of datasets and instrument of public participation. The website allows discovery of datasets in several ways: general search, tags/keywords, categories, and rating. The public interacts with government through a voting mechanism made popular in social networking sites. The public gives feedback by making datasets more popular or requesting datasets not yet available. Additionally, the DataSF.org team works with city departments to make available additional data sources that have not been freely presented to the public in the past.
FAQs
What is open data?
Open Data is a philosophy and practice requiring that certain data are freely available to everyone, without restrictions from copyright, patents or other mechanisms of control. The majority of government data actually belongs to the citizenry thus government is the stewards of data not the owner of it.
Why open government?
Government should be transparent, participatory, and collaborative. Transparency promotes accountability of Government by its constituents. Public participation enhances the Government’s effectiveness and through collaboration improves the quality of its service delivery.
What is the value proposition to the public?
The public is given the opportunity for discovery of information, knowledge and innovation. Increasing access to government datasets encourages creation of innovative civic tools and applications. Transportation, recreation, health, and safety applications have been built using Government data enriching the lives of the public and at no cost to Government. Another example of open government data was the Human Genome project which benefits humanity beyond the borders of any jurisdiction.
What is the value proposition to the government?
Departments gain increased public’s awareness of their work and can produce a greater trust and appreciation for department missions, their roles, and their overall performance in the service of the City and bettering the lives of the public.
Process
1. Understand your maturity and options
Prior to issuing an open data executive directive or legislation, it would be prudent to do some socialization of open government and open data. Departments may not see the benefit of opening its data to public inspection. When describing the benefits concrete examples of benefits experience elsewhere may help. One strategy is to explain that public collaboration may result in the department being able to offer more services at the same or less cost.
Departments may also not currently have the standard operating procedures that encourage machine readable data. Explanation and policy should be created around acceptable data formats. There may also be gaps in infrastructure allowing the departments to disseminate the data to the public or to a centralized location. Depending on your municipality the effort may require additional expenditure.
As with any new program, personnel will likely be needed, at least part-time, to function as data stewards. There should be a representative in each department which is tasked with ensuring that the department data is available and updated regularly. Depending on the size of your municipality a City-wide data steward who is the subject matter expert on open data will work with the other representatives to ensure compliance with the policy.
2. Decide on your data delivery approach
A website should be created to explain the open data effort for internal and external stakeholders but also as a “one-stop-shop” for finding the datasets. However, there are at least two approaches to housing the datasets: a centralized repository or links to each departmental repository.
The benefit of having a central repository is that it is easier to manage the infrastructure serving the datasets. In a centralized repository, each department would upload its dataset and that dataset would then be served to the public. This approach has the added benefit of creating a secure location for the public to interact with the government data. This central repository could also be in a hosted location thereby further separating the public from sensitive internal data and applications. In the first phase of the open data project, once a dataset is uploaded to the central repository it will be static.
The other option is to link to the datasets from the website. This allows more departmental control and the potential for more current datasets. This approach could result in links from the website being broken as the locations of repositories or datasets move.
3. Evolution of open data
The City-wide data steward will be work with each department’s representative on releasing more datasets over time. It may work well to have technical people as departmental representatives in which a collaborative atmosphere is fostered. The evolution of open data is one in which publication of datasets becomes an automated process. This will result in more datasets released in a timely manner.
There will need to be a compliance organization that verifies that departments are complying with the Executive order or legislation. This organization may also be the sound board for future changes due to changes in technology, data formats, or security. Standards on interdepartmental naming conventions and data types may facilitate public interaction with the datasets. Standardization allows for more tools to be built with less development time and effort.
Public feedback will be crucial to the evolution of the open data initiative. The public is the consumer of the datasets and thus their needs need to be taken into account when evolving the open data program.
Stakeholders
- Departments of the City and County of San Francisco
- Citizens of the City and County of San Francisco
- Visitors of the City and County of San Francisco
Policy
Executive Directive 09-06
Open Data
October 21, 2009
By virtue of the power and authority vested in me by Section 3.100 of the San Francisco Charter to provide administration and oversight of all departments and governmental units in the executive branch of the City and County of San Francisco, I do hereby issue this Executive Directive to become effective immediately:
- Benefits of the Open Data Directive
- The City declares its commitment to transparency in government data.
This Directive will enhance open government, transparency, and accountability by improving access to City data that adheres to privacy and security policies. Data which often resides in technology systems is unique from information like documents, emails and calendars in that it is structured and can be used by other computer applications for analysis or new uses such as mapping. This Directive establishes a one-stop destination for all approved City data that will help constituents make better use of information. This new ease of access will lead to innovation in how residents interact with government, resulting in social and economic benefits for the City. The City and County of San Francisco will be able to engage our innovative high-tech workforce by releasing data, a key component of San Francisco’s future economic development. By providing government data that adheres to privacy and security policies, San Francisco’s world class technology community is given the platform from which to create useful civic tools, all at no cost to City government. By bringing City data and San Francisco’s entrepreneurs together, we can effectively leverage existing resources to stimulate industry, create jobs and highlight San Francisco’s creative culture and attractiveness as a place to live and work. Finally, the City and
County of San Francisco’s technology presence will begin to reflect that of our world class, cutting edge private technology sector, and help us better engage the wealth of knowledge and skills of our local community.
- a. All datasets determined to be accessible to the public now and in the future shall be made available through DataSF.org.
b. All Department Heads or their designees shall strive to publish all datasets under their authority consistent with the DataSF authorization policy. Detailed information on how to update DataSF.org is available here
c. All Department Heads or their designees shall conduct quarterly reviews of their progress on providing access to datasets requested by the public through DataSF.org. The results of this review shall be sent to the Mayor’s Office.
Resources
Pligg zip file ( Technology powering DataSF.org you’ll need to download and install pligg first version 1.0.0 )
State CIO group white paper on open data
Media announcement of DataSF on TechCrunch
App Showcase
Media announcement of App Showcase
Dept Scorecard
Authorization Policy
DataSF Terms of Use
How to submit a new dataset to DataSF (includes overall process map)
Toolkit
Download Open Data Toolkit in pdf format
Download Adobe Reader to view toolkit
[...] We will be adding more toolkits to PolicySF in the weeks and months ahead. The site currently features a number of San Francisco policies, including Healthy San Francisco, our plastic bag ban, San Francisco Promise, JobsNow and our open data policy. [...]
[...] Additionally, we are asking local communities to share their own good policy ideas with visitors to the site. Perhaps someone in Chicago or Tulsa has figured out a creative solution to a problem and wants to let everyone know how to replicate it. We will be adding more toolkits to PolicySF in the weeks and months ahead. The site currently features a number of San Francisco policies, including Healthy San Francisco, our plastic bag ban, San Francisco Promise, JobsNow and our open data policy. [...]
[...] We will be adding more toolkits to PolicySF in the weeks and months ahead. The site currently features a number of San Francisco policies, including Healthy San Francisco, our plastic bag ban, San Francisco Promise, JobsNow and our open data policy. [...]
[...] We will be adding more toolkits to PolicySF in the weeks and months ahead. The site currently features a number of San Francisco policies, including Healthy San Francisco, our plastic bag ban, San Francisco Promise, JobsNow and our open data policy. [...]