Development Experience Clearinghouse DEC API Help and Documentation

About the DEC

The Development Experience Clearinghouse (DEC) is the largest online resource for USAID funded technical and project materials for international development, with assets dating back to 1945. You are able to access information on over 190,000 USAID technical and program documents with more than 155,500 available for electronic download and growing everyday.

The purpose of the DEC is to strengthen USAID's development projects, activities, and programs by making these development experience documents available to USAID offices and mission staff, PVO's, NGO's, universities and research institutions, developing countries, and the public worldwide.

About the API

The DEC API can assist you in extracting meta-data and records from the DEC. The DEC website (http://dec.usaid.gov), with its rich user interface is optimized for search and discovery, but not the extraction of large amounts of data. Through the website, you can explore the DEC’s offerings and become familiar with the content and search syntax of the DEC. After a certain amount of exploration, you may discover that you need to extract large amounts of data. That’s where the DEC API comes in. It is designed to provide a mechanism to retrieve large amounts of data.

This page will provide basic instructions on how to call and interact with the inputs and outputs of the DEC API. In order to understand the offerings of the API, we expect that you have some experience with programming including constructing an URL, web methods calls, and http requests.

The API consist of three web points that provide distinct mechanisms to interface with the DEC. The points of access are qsearch.ashx, queries.svc, and request.axpx. Each contains their own methods to retrieve information from the DEC.

About the Dataset

The dataset includes the meta-data associated with each document record in the database. The meta-data is derived from the content of the medium and provides information such as the object's URL, title, publication date, authors, document types, and geographical regions. Please refer to the data dictionary for more information on what is returned by the API.

qsearch.ashx

Qsearch will immediately return the result set in the specified format as a downloadable feed for search results under 2,000 records. Any queries more than the record limit will return a "result set too large error."

Response Format:

CSV, JSON, XML

Response Type:

HTTPS GET

Authentication:

None

Required Variables:

q= String - Base64 URL Encoded Search Query for the DEC Database. If the field is ignored then qsearch will redirect back to help.html.
To form a valid Base64 URL encoded string, use these sites to assist in building your query.
"Search string" -> Base64 -> URLEncode -> Parameter -> Request -> Output
"Search string" -> U2VhcmNoIHN0cmluZw== -> U2VhcmNoIHN0cmluZw%3D%3D -> q=U2VhcmNoIHN0cmluZw%3D%3D -> http://dec.usaid.gov/api/qsearch.ashx?q=U2VhcmNoIHN0cmluZw%3D%3D

Optional Variables:

rtype= String Enumeration - Can be either CSV, JSON, or XML. Default is XML. If an invalid entry is supplied then the application will use default output type of XML.
Enumeration -> Parameter -> Output
CSV -> rtype=CSV -> http://dec.usaid.gov/api/qsearch.ashx?q=U2VhcmNoIHN0cmluZw%3D%3D&rtype=CSV

How to build your query:

Lets get Evaluations from the DEC using the search notation (documents.web_collection:("recent evaluations")) and output as a CSV file.

Take (documents.web_collection:("recent evaluations")) and convert it to Base64.

Take KGRvY3VtZW50cy53ZWJfY29sbGVjdGlvbjooInJlY2VudCBldmFsdWF0aW9ucyIpKQ== and URL encode it.

Take KGRvY3VtZW50cy53ZWJfY29sbGVjdGlvbjooInJlY2VudCBldmFsdWF0aW9ucyIpKQ%3D%3D and set as a 'q' parameter for the URL: http://dec.usaid.gov/api/qsearch.ashx

http://dec.usaid.gov/api/qsearch.ashx?q=KGRvY3VtZW50cy53ZWJfY29sbGVjdGlvbjooInJlY2VudCBldmFsdWF0aW9ucyIpKQ%3D%3D will return its results using its default output format as XML.

Append the string with the parameter and value notation of &rtype=CSV.
http://dec.usaid.gov/api/qsearch.ashx?q=KGRvY3VtZW50cy53ZWJfY29sbGVjdGlvbjooInJlY2VudCBldmFsdWF0aW9ucyIpKQ%3D%3D&rtype=CSV will now output as a CSV file.

Now you have a URL that will get all of the DEC's meta-data for the Evaluations as a CSV file.

Queries.svc

Queries.svc is a Mircosoft Windows Communication Foundation Service that is hosted by IIS. It provides an object to call methods that can return recourd counts, results below 2,000 records, and request larger sets in the same way Request.aspx provides. These methods can be referenced in an .NET application. Search the web for more information on WCF Services.

Response Type:

Mircosoft Windows Communication Foundation Service

Authentication:

None

Methods:

int CountResults(string email, string query, string version)

Counts the number of records in the search query. If the query does not parse or there was an internal error then zero(0) is returned.

email - requestor's email address
query - DEC search string
version - Reserved for future releases. Set to NULL the value is ignored.

Returns the number of records found in the DEC using the query string.

string Request(string email, string query, string version, string format)

Appends the request into the API's processing queue and sends a confirmation email on the success of the request.

email - requestor's email address
query - DEC search string
version - Reserved for future releases. Set to NULL the value is ignored.
format - Can be either CSV, JSON, or XML. If the value does not parse then XML is assumed.

Returns the confirmation or error text of the transaction.

string Search(string email, string query, string version, string format)

Searches the DEC for an immediate response. If the record count is larger than 2,000 records then a "result set too large error." is returned.

email - requestor's email address
query - DEC search string
version - Reserved for future releases. Set to NULL the value is ignored.
format - Can be either CSV, JSON, or XML. If the value does not parse then XML is assumed.

Returns the string representation of the result set in the output format provided. See the Output Samples for more information.

Request.aspx

Request.aspx will request and record the requestor's email address, search query, and output format and when the submit button is clicked, it appends the request into the API's processing queue. Immediately the site will send an email to the requestor's address to provide a confirmation of the request. Then during the site's off peek hours, the API will generate the result and send an additional email to the requestor with a link to the data in the requested format as a downloadable file from the site.

Links are stored on the server for 30 days. They are not renewed or refreshed automatically and its content stays the same. If you need to refresh the data then you should complete a new request. If you need a link to our site to remain pernment then contact the DEC webmaster at decwebmaster@usaid.gov and submit your request in the email.

Response Type:

Forms response

Authentication:

Requires email address to process the request.

Form fields:

Email *

Required - Requestor's email address to send the confirmation and result link to when the operation is completed.

Search Query *

Required - Search Query for the DEC Database

Check query

Optional function to evaluate your search query to a count of the records. Use this to make sure that your search query is going to return an expected number of results.

Output Format

Defaults to XML - A dropdown box that allows the user to choose a result format between XML, JSON, and CSV.

Submit

Checks the query for valid values and if they are valid then enters the values to the queue for processing after hours. Two emails will be generated afterward submittion. The first will be send immediatly to the requestor and the second will be send when the process is complete.

Reset

Reset the form fields for a new query. Note that Reset does not clear the confirmation or error text in the form.

Output:

The page will return a confirmation text and a confirmation email to the user. You can use the confirmation number to track insidents with the API to the decwebmaster at decwebmaster@usaid.gov

Output:

The output depends upon the user specified "rtype" or output type of XML, JSON, or CSV. The schema contains the search term, the number of records found, and a listing of all matching records in the database.

Sample CSV Return

Sample Property,Another Sample Property\n
"Sample Value","Another Sample Value | And a multi-value"\n

Sample Property	Another Sample Property
Sample Value	Another Sample Value \| And a multi-value

Note: Multivalues in the CSV format is delimited by space then pipe and then space again. (' | ').

Sample JSON Return

{

"SearchTerm": "Sample",
"SearchDate": "2014-01-01T11:59:59.0000000-04:00",
"RecordsFound": "68",
"Records": [

{

"Sample Property":
{

"name": "Sample Property",
"value": ["Sample Value"]

"Another Sample Property":
{

"name": "Another Sample Property",
"value": ["Another Sample Value", "And a multi-value"]

},
...

},
{

...

},
...

]

}

Sample XML Return

<SearchTerm>Sample</SearchTerm>
<SearchDate>2014-01-01T11:59:59.0000000-04:00</SearchDate>
<RecordsFound>68</RecordsFound>
<Records>

<Value>Sample Value</Value>

</Property>

<Value>Another Sample Value</Value>
<Value>And a multi-value</Value>

</Property>
...

</Record>
<Record>

...

</Record>
...

Data Dictionary

Header information

searchTerm
searchDate
RecordsFound

Property name	Data type	Sample value	Description
searchTerm	String	Forward Evaluations	Text value of the search query executed against the DEC.
searchDate	Date	2014-01-01T11:59:59.0000000-04:00	Exact date and time the search query started its execution against the DEC.
RecordsFound	Integer	68	Indicates the number of records found with the search query and the number of record nodes in the output.

Column ID's

Abstract
Ancillary_Data
Bibliographic_Type
ContentType
Contract_Grant_Number
Credit
Date_Resource_Created
Description
Descriptors_Topical
Descriptors_Geographic
File
File_Size
Inst_Author
Inst_Publisher
Inst_Sponsor
Language
Mime_Type
New_Thesaurus_Terms
Notes
Personal_Author
Primary_Subject
Publication_Date_Freeform
Related_Doc_Links
Report_Number
Series_Title
Title
Title_Translated
Unique_ID
URI
USAID_Geography
USAID_Project_Number

Property name	Data type	Sample value	Description
Abstract	String	Summarizes mid-term evaluation of a project...	The summary of the artifact.
Ancillary_Data	String	Period of report: August 2003-July 2004	Can be anything from the document that doesn't fit in a predefined field; also information that is not available in the document.
Bibliographic_Type	String	Reference Document	Bibliographic Code and Name; Indicates the type of artifact.
ContentType	String	Documents	Identifies the content type of the record. Can be Documents, Audio, Video, etc.
Contract_Grant_Number	String	PCE-I-00-00-00000-00	Number of the A.LD. contract, grant, PASA, or RSSA which funded the production of the artifact.
Credit	String	Director: James Cameron	The people who are responsible for creating the object: i.e. photographer, director, writer, interviewer, interviewee, actor, etc.
Date_Resource_Created	Date	1/1/1986 12:00:00 AM	Displayed publication date on the document; Date the document was (is to be) published; submission of dissertation or thesis; defined format.
Description	String	Video is in color and contains sound	Describes the artifact for its look in feel. Relates to Multimedia content types.
Descriptors_Topical	String	HIV/AIDS	Field contains primary descriptors which reflect the subject content of the artifact.
Descriptors_Geographic	String	Latin America	Geopolitical descriptors from the USAID thesaurus. This is the geographic topic of the asset.
File	URL	http://pdf.usaid.gov/pdf_docs/PNAAA000.pdf	The actual artifact document.
File_Size	String	300 KB	The file size in KB.
Inst_Author	String	000035 - U.S. Agency for International Development (USAID)	Authoring organization name.
Inst_Publisher	String	012781 - USAID. Knowledge Services Center (KSC)	Publishing organization name.
Inst_Sponsor	String	000035 - U.S. Agency for International Development (USAID)	Sponsoring organization name.
Language	String	English	Identifies the language of the multimedia file. Identifies the language of the text of the document being analyzed. If more than one language is used, enter the appropriate codes in alphabetical order, separated by a comma and space.
Mime_Type	String	application/pdf	Mime type; the internet media type identifier.
New_Thesaurus_Terms	String	Interim evaluation	Terms NOT included in the"A.I.D. Thesaurus" are entered separately here to describe concepts not adequately covered by existing descriptors. Field also contains proposed descriptors.
Notes	String	Additional information can be found at...	Area for freeform notes.
Personal_Author	String	Shah, Rajiv	Persons responsible for authorship, includes editors and compilers.
Primary_Subject	String	Development organizations	Primary Subject; A major subject category is assigned to all technical document records entered after August 1985 and to all project documents entered after February 1987. The subject is derived from the "A.I.D. Thesaurus" list of Major Subjects.
Publication_Date_Freeform	String	Spring 1987	Displayed publication date on the document; Date the document was (is to be) published; submission of dissertation or thesis; unstructured format.
Related_Doc_Links	String	http://pdf.usaid.gov/pdf_docs/PNAAA001.pdf	List of other documents that relate to this artifact.
Report_Number	String	AS-0-0	Alphanumeric number assigned to a document by the creators.
Series_Title	String	Population technical assistance project occasional paper, no. 6	Records the title of the publication series and the part.
Title	String	USAID HIV/AIDS segment : overview -- final	The title and subtitle of an artifact.
Title_Translated	String	USAID HIV/AIDS segment : overview -- final	English translation of the Document Title.
Unique_ID	String	PN-AAA-000	Unique identifier. Also used to indicate that the cataloguer has selected a particular document for cataloguing since they are the ones applying the DocID. PDs are for non-technical documents, PNs technical, and PCs are non-USAID documents, typically found on the web.
URI	URL	http://dec.usaid.gov/dec/content/Detail.aspx?ctID=ODVhZjk4NWQtM2YyMi00YjRmLTkxNjktZTcxMjM2NDBmY2Uy&rID=0	URL to the details record page in the DEC.
USAID_Geography	String	Ecuador	Geographic Area Name as defined by USAID; Related to code field. This is where the funding for the asset came from.
USAID_Project_Number	String	1234567	Concatenation of the Project and Sub-project numbers. Seven or Nine digits.

Development Experience Clearinghouse DEC API Help and Documentation

Table of Contents

Introduction

About the DEC

About the API

About the Dataset

qsearch.ashx

Response Format:

Response Type:

Authentication:

Required Variables:

Optional Variables:

How to build your query:

Queries.svc

Response Type:

Authentication:

Methods:

int CountResults(string email, string query, string version)

string Request(string email, string query, string version, string format)

string Search(string email, string query, string version, string format)

Request.aspx

Response Type:

Authentication:

Form fields:

Email *

Search Query *

Check query

Output Format

Submit

Reset

Output:

Output:

Sample CSV Return

Sample JSON Return

Sample XML Return

Data Dictionary

Header information

Column ID's