Development Experience Clearinghouse DEC API Help and Documentation

Table of Contents

Introduction

About the DEC

The Development Experience Clearinghouse (DEC) is the largest online resource for USAID funded technical and project materials for international development, with assets dating back to 1945. You are able to access information on over 190,000 USAID technical and program documents with more than 155,500 available for electronic download and growing everyday.

The purpose of the DEC is to strengthen USAID's development projects, activities, and programs by making these development experience documents available to USAID offices and mission staff, PVO's, NGO's, universities and research institutions, developing countries, and the public worldwide.

About the API

The DEC API can assist you in extracting meta-data and records from the DEC. The DEC website (http://dec.usaid.gov), with its rich user interface is optimized for search and discovery, but not the extraction of large amounts of data. Through the website, you can explore the DECís offerings and become familiar with the content and search syntax of the DEC. After a certain amount of exploration, you may discover that you need to extract large amounts of data. Thatís where the DEC API comes in. It is designed to provide a mechanism to retrieve large amounts of data.

This page will provide basic instructions on how to call and interact with the inputs and outputs of the DEC API. In order to understand the offerings of the API, we expect that you have some experience with programming including constructing an URL, web methods calls, and http requests.

The API consist of three web points that provide distinct mechanisms to interface with the DEC. The points of access are qsearch.ashx, queries.svc, and request.axpx. Each contains their own methods to retrieve information from the DEC.

About the Dataset

The dataset includes the meta-data associated with each document record in the database. The meta-data is derived from the content of the medium and provides information such as the object's URL, title, publication date, authors, document types, and geographical regions. Please refer to the data dictionary for more information on what is returned by the API.

Back to Top

qsearch.ashx

Qsearch will immediately return the result set in the specified format as a downloadable feed for search results under 2,000 records. Any queries more than the record limit will return a "result set too large error."

Response Format:

CSV, JSON, XML

Response Type:

HTTPS GET

Authentication:

None

Required Variables:

Optional Variables:

How to build your query:

Lets get Evaluations from the DEC using the search notation (documents.web_collection:("recent evaluations")) and output as a CSV file.

Take (documents.web_collection:("recent evaluations")) and convert it to Base64.

Take KGRvY3VtZW50cy53ZWJfY29sbGVjdGlvbjooInJlY2VudCBldmFsdWF0aW9ucyIpKQ== and URL encode it.

Take KGRvY3VtZW50cy53ZWJfY29sbGVjdGlvbjooInJlY2VudCBldmFsdWF0aW9ucyIpKQ%3D%3D and set as a 'q' parameter for the URL: http://dec.usaid.gov/api/qsearch.ashx

http://dec.usaid.gov/api/qsearch.ashx?q=KGRvY3VtZW50cy53ZWJfY29sbGVjdGlvbjooInJlY2VudCBldmFsdWF0aW9ucyIpKQ%3D%3D will return its results using its default output format as XML.

Append the string with the parameter and value notation of &rtype=CSV.
http://dec.usaid.gov/api/qsearch.ashx?q=KGRvY3VtZW50cy53ZWJfY29sbGVjdGlvbjooInJlY2VudCBldmFsdWF0aW9ucyIpKQ%3D%3D&rtype=CSV will now output as a CSV file.

Now you have a URL that will get all of the DEC's meta-data for the Evaluations as a CSV file.

Back to Top

Queries.svc

Queries.svc is a Mircosoft Windows Communication Foundation Service that is hosted by IIS. It provides an object to call methods that can return recourd counts, results below 2,000 records, and request larger sets in the same way Request.aspx provides. These methods can be referenced in an .NET application. Search the web for more information on WCF Services.

Response Type:

Mircosoft Windows Communication Foundation Service

Authentication:

None

Methods:

int CountResults(string email, string query, string version)

Counts the number of records in the search query. If the query does not parse or there was an internal error then zero(0) is returned.

string Request(string email, string query, string version, string format)

Appends the request into the API's processing queue and sends a confirmation email on the success of the request.

string Search(string email, string query, string version, string format)

Searches the DEC for an immediate response. If the record count is larger than 2,000 records then a "result set too large error." is returned.

Back to Top

Request.aspx

Request.aspx will request and record the requestor's email address, search query, and output format and when the submit button is clicked, it appends the request into the API's processing queue. Immediately the site will send an email to the requestor's address to provide a confirmation of the request. Then during the site's off peek hours, the API will generate the result and send an additional email to the requestor with a link to the data in the requested format as a downloadable file from the site.

Links are stored on the server for 30 days. They are not renewed or refreshed automatically and its content stays the same. If you need to refresh the data then you should complete a new request. If you need a link to our site to remain pernment then contact the DEC webmaster at decwebmaster@usaid.gov and submit your request in the email.

Response Type:

Forms response

Authentication:

Requires email address to process the request.

Form fields:

Email *

Required - Requestor's email address to send the confirmation and result link to when the operation is completed.

Search Query *

Required - Search Query for the DEC Database

Check query

Optional function to evaluate your search query to a count of the records. Use this to make sure that your search query is going to return an expected number of results.

Output Format

Defaults to XML - A dropdown box that allows the user to choose a result format between XML, JSON, and CSV.

Submit

Checks the query for valid values and if they are valid then enters the values to the queue for processing after hours. Two emails will be generated afterward submittion. The first will be send immediatly to the requestor and the second will be send when the process is complete.

Reset

Reset the form fields for a new query. Note that Reset does not clear the confirmation or error text in the form.

Output:

The page will return a confirmation text and a confirmation email to the user. You can use the confirmation number to track insidents with the API to the decwebmaster at decwebmaster@usaid.gov

Back to Top

Output:

The output depends upon the user specified "rtype" or output type of XML, JSON, or CSV. The schema contains the search term, the number of records found, and a listing of all matching records in the database.

Sample CSV Return

Sample Property,Another Sample Property\n
"Sample Value","Another Sample Value | And a multi-value"\n

Sample PropertyAnother Sample Property
Sample ValueAnother Sample Value | And a multi-value

Note: Multivalues in the CSV format is delimited by space then pipe and then space again. (' | ').

Sample JSON Return

{
"SearchTerm": "Sample",
"SearchDate": "2014-01-01T11:59:59.0000000-04:00",
"RecordsFound": "68",
"Records": [
{
"Sample Property":
{
"name": "Sample Property",
"value": ["Sample Value"]
},
"Another Sample Property":
{
"name": "Another Sample Property",
"value": ["Another Sample Value", "And a multi-value"]
},
...
},
{
...
},
...
]
}

Sample XML Return

<Search>
<SearchTerm>Sample</SearchTerm>
<SearchDate>2014-01-01T11:59:59.0000000-04:00</SearchDate>
<RecordsFound>68</RecordsFound>
<Records>
<Record>
<Property name="Sample Property">
<Value>Sample Value</Value>
</Property>
<Property name="Another Sample Property">
<Value>Another Sample Value</Value>
<Value>And a multi-value</Value>
</Property>
...
</Record>
<Record>
...
</Record>
...
<Records>
<Search>

Back to Top

Data Dictionary

Header information

Property nameData typeSample valueDescription
searchTermStringForward EvaluationsText value of the search query executed against the DEC.
searchDateDate2014-01-01T11:59:59.0000000-04:00Exact date and time the search query started its execution against the DEC.
RecordsFoundInteger68Indicates the number of records found with the search query and the number of record nodes in the output.

Column ID's

Property nameData typeSample valueDescription
AbstractStringSummarizes mid-term evaluation of a project...The summary of the artifact.
Ancillary_DataStringPeriod of report: August 2003-July 2004Can be anything from the document that doesn't fit in a predefined field; also information that is not available in the document.
Bibliographic_TypeStringReference DocumentBibliographic Code and Name; Indicates the type of artifact.
ContentTypeStringDocumentsIdentifies the content type of the record. Can be Documents, Audio, Video, etc.
Contract_Grant_NumberStringPCE-I-00-00-00000-00Number of the A.LD. contract, grant, PASA, or RSSA which funded the production of the artifact.
CreditStringDirector: James CameronThe people who are responsible for creating the object: i.e. photographer, director, writer, interviewer, interviewee, actor, etc.
Date_Resource_CreatedDate1/1/1986 12:00:00 AMDisplayed publication date on the document; Date the document was (is to be) published; submission of dissertation or thesis; defined format.
DescriptionStringVideo is in color and contains soundDescribes the artifact for its look in feel. Relates to Multimedia content types.
Descriptors_TopicalStringHIV/AIDSField contains primary descriptors which reflect the subject content of the artifact.
Descriptors_GeographicStringLatin AmericaGeopolitical descriptors from the USAID thesaurus. This is the geographic topic of the asset.
FileURLhttp://pdf.usaid.gov/pdf_docs/PNAAA000.pdfThe actual artifact document.
File_SizeString300 KBThe file size in KB.
Inst_AuthorString000035 - U.S. Agency for International Development (USAID)Authoring organization name.
Inst_PublisherString012781 - USAID. Knowledge Services Center (KSC)Publishing organization name.
Inst_SponsorString000035 - U.S. Agency for International Development (USAID)Sponsoring organization name.
LanguageStringEnglishIdentifies the language of the multimedia file. Identifies the language of the text of the document being analyzed. If more than one language is used, enter the appropriate codes in alphabetical order, separated by a comma and space.
Mime_TypeStringapplication/pdfMime type; the internet media type identifier.
New_Thesaurus_TermsStringInterim evaluationTerms NOT included in the"A.I.D. Thesaurus" are entered separately here to describe concepts not adequately covered by existing descriptors. Field also contains proposed descriptors.
NotesStringAdditional information can be found at...Area for freeform notes.
Personal_AuthorStringShah, RajivPersons responsible for authorship, includes editors and compilers.
Primary_SubjectStringDevelopment organizationsPrimary Subject; A major subject category is assigned to all technical document records entered after August 1985 and to all project documents entered after February 1987. The subject is derived from the "A.I.D. Thesaurus" list of Major Subjects.
Publication_Date_FreeformStringSpring 1987Displayed publication date on the document; Date the document was (is to be) published; submission of dissertation or thesis; unstructured format.
Related_Doc_LinksStringhttp://pdf.usaid.gov/pdf_docs/PNAAA001.pdfList of other documents that relate to this artifact.
Report_NumberStringAS-0-0Alphanumeric number assigned to a document by the creators.
Series_TitleStringPopulation technical assistance project occasional paper, no. 6Records the title of the publication series and the part.
TitleStringUSAID HIV/AIDS segment : overview -- finalThe title and subtitle of an artifact.
Title_TranslatedStringUSAID HIV/AIDS segment : overview -- finalEnglish translation of the Document Title.
Unique_IDStringPN-AAA-000Unique identifier. Also used to indicate that the cataloguer has selected a particular document for cataloguing since they are the ones applying the DocID. PDs are for non-technical documents, PNs technical, and PCs are non-USAID documents, typically found on the web.
URIURLhttp://dec.usaid.gov/dec/content/Detail.aspx?ctID=ODVhZjk4NWQtM2YyMi00YjRmLTkxNjktZTcxMjM2NDBmY2Uy&rID=0URL to the details record page in the DEC.
USAID_GeographyStringEcuadorGeographic Area Name as defined by USAID; Related to code field. This is where the funding for the asset came from.
USAID_Project_NumberString 1234567Concatenation of the Project and Sub-project numbers. Seven or Nine digits.

Back to Top