NPR Blogs: Inside NPR.org http://www.npr.org/blogs/inside/ Inside NPR.org en Copyright 2009 Thu, 25 Jun 2009 14:02:18 -0500 http://www.sixapart.com/movabletype/ http://blogs.law.harvard.edu/tech/rss Upcoming Changes to the API In the next month or so, we will be making some significant changes to NPR.org. Some of these changes are visual, while others are architectural. As a result, there will likely be an impact on the API. That said, we did put in a lot of effort to make the system as backward compatible as possible to ensure that API users would be as minimally affected as possible. I will post again to this blog soon with more details on the changes. In the meantime, here are some high-level descriptions of what to expect: 1. There will be changes to our topic structure resulting in some topics being eliminated, others being added, and others being renamed. For any changes to existing topics, there will be redirects to corresponding topics that will be maintained for a reasonable period of time to ensure backward compatibility. Our goal is to ensure that any applications that are dependent on specific topics existing will continue to work. 2. There will be some nodes and parameters added to the API output for NPRML. These will largely be to support the new features on NPR.org. They should not break any applications dependent on NPRML unless those applications require that these additional elements do not exist. 3. There will be new products and extensions added to the API. These will not adversely affect any current API calls. As I mentioned earlier, I will publish to this blog again with more details on the changes as we draw closer. In the meantime, please provide any feedback or concerns about this in the comments section for this post. --Daniel Jacobson In the next month or so, we will be making some significant changes to NPR.org. Some of these changes are visual, while others are architectural. As a result, there will likely be an impact on the API. That said, we did put in a lot of effort to make the system as backward compatible as possible to ensure that API users would be as minimally affected as possible.

I will post again to this blog soon with more details on the changes. In the meantime, here are some high-level descriptions of what to expect:

1. There will be changes to our topic structure resulting in some topics being eliminated, others being added, and others being renamed. For any changes to existing topics, there will be redirects to corresponding topics that will be maintained for a reasonable period of time to ensure backward compatibility. Our goal is to ensure that any applications that are dependent on specific topics existing will continue to work.

2. There will be some nodes and parameters added to the API output for NPRML. These will largely be to support the new features on NPR.org. They should not break any applications dependent on NPRML unless those applications require that these additional elements do not exist.

3. There will be new products and extensions added to the API. These will not adversely affect any current API calls.

As I mentioned earlier, I will publish to this blog again with more details on the changes as we draw closer. In the meantime, please provide any feedback or concerns about this in the comments section for this post.
--Daniel Jacobson

]]> <p><a href="http://www.npr.org/blogs/inside/2009/06/upcoming_changes_to_the_api.html#email">&raquo; E-Mail This</a>&nbsp;&nbsp;&nbsp;&nbsp; <a href="http://del.icio.us/post?url=http://www.npr.org/blogs/inside/2009/06/upcoming_changes_to_the_api.html">&raquo; Add to Del.icio.us</a> </p> http://www.npr.org/blogs/inside/2009/06/upcoming_changes_to_the_api.html?ft=1&f=91000411 http://www.npr.org/blogs/inside/2009/06/upcoming_changes_to_the_api.html?ft=1&f=91000411 API API Thu, 25 Jun 2009 14:02:18 -0500 Search, Matches and Self Service If you have used our search recently, you may have noticed that we just launched a 'new' search in beta. You can either follow the link from the search page or you can try it by clicking here.

So what's the big deal you ask? Visually, we made changes to have cleaner look while getting the results more prominently positioned on the page.   Behind the scenes the technology is completely different. The new search is powered by the Google Search Appliance. While our previous search tool has similar potential ability to yield accurate results, it required a high degree of technical expertise to tune. One of the core philosophies in NPR Digital Media's technology team is that we want to be a partner, not a bottleneck to innovation. As part of this we embrace the idea we call 'Self Service' -- the antithesis to maintaining a technology fiefdom. The idea being the more we can provide empowering tools to our colleagues, the more we can accomplish as a team. Prior to the inventions of the lighter and matches, starting a fire was a cumbersome affair, typically involving a tinderbox, flint and a piece of steel. In modern day we usually take making fire for granted -- not because we all have become experts in the discipline, but rather because we have self service tools that work really well. So that is the root of our approach: implement smart, maintainable tools that are easy for people to use.

So it is this same spirit of self service that led to the selection of the search appliance. In this case there was no need to build it ourselves, as other companies had invested quite a lot in making solid search tools. While we think Google is really smart with its search algorithms, even more appealing was the ease of tuning via the appliance's interface. A colleague of ours Javaun Moradi is in charge of search (among many other things). Without making any changes to our code or critical configurations he is able to easily make changes using the GSA interface to help ensure we are indexing and surfacing the results that are desired. Using the variety of information and meta-data available about our content, rules can be defined to bias towards more relevant pages, and make sure to exclude redundant or unnecessary items from the results. Even within the first 24 hours of the tool being up in beta, he informs me that he has already made several tuning changes to help surface information about our shows while also surfacing breaking news.

Another aspect we especially like is the ability of the appliance to render its results in XML. While a suggested implementation is to use XSLT directly on the box to yield result pages, we appreciate the flexibility to make service calls to the appliance and then work with the very clean, portable XML results. Currently we are rendering out the XML results as search result pages using PHP -- which mirrors the architecture we use for the rest of the site. We expect in the future we will be able to use this to better integrate other content and features with search, and search with other features.

This leads us to why we are launching this new search in 'beta'. While we have done some tuning and configuration we are still working to get it right. By putting this tool out in a preliminary beta we can watch to see the queries it is getting and tune it to make it better. On the web, seeing how you all actually use our website is the most authoritative way to judge what is working and what isn't. Everyday is an opportunity to improve upon what we did yesterday. One example of this is that we see many searches that people mistakenly believe NPR produce, such as "This American Life" or "A Prairie Home Companion". By seeing that users are making these searches, we can make sure appropriate results are showing up. So whether you are searching for a story you heard this morning, or want to find those performances at Bob Boilen's tiny desk -- hopefully we get you what you want. We currently anticipate moving it out of 'beta' and as our primary search tool later this summer as we announce some other changes to our digital media tools and products.

Please share any observations or feedback below, or via this comment tool we setup specifically for the new search. We anticipate numerous improvements in the months ahead.

Happy Searching.

-- Zach Brand

]]> <p><a href="http://www.npr.org/blogs/inside/2009/06/search_and_matches.html#email">&raquo; E-Mail This</a>&nbsp;&nbsp;&nbsp;&nbsp; <a href="http://del.icio.us/post?url=http://www.npr.org/blogs/inside/2009/06/search_and_matches.html">&raquo; Add to Del.icio.us</a> </p> http://www.npr.org/blogs/inside/2009/06/search_and_matches.html?ft=1&f=91000411 http://www.npr.org/blogs/inside/2009/06/search_and_matches.html?ft=1&f=91000411 Technology Wed, 17 Jun 2009 12:20:26 -0500 NPR's API Rights Management One of the things that I am most commonly asked about regarding the NPR API is rights management. Because we are distributing content to unknown destinations, it is critical to make sure the API itself can control what gets offered and to whom. To handle these kinds of issues, we built a robust permissions and rights management system into the API. But that is not enough. Rights management starts with contracts and ensuring that the content is tagged appropriately. Without these steps, the rights management system cannot accurately withhold the content that is not allowed to be distributed. So, here is a breakdown of the steps we went through and the systems we built to handle rights in our API. Contracts Before launching the API, we spent a lot of time with our legal team reviewing existing contracts and our rights tagging system. Based on this review, we determined that a few changes needed to be made to the rights tagging system, but there were quite a few restrictions on what could be offered through the API. One interesting example is Fresh Air. Fresh Air is a program produced by WHYY and distributed on the radio by NPR. NPR is also responsible for displaying the content on NPR.org and is allowed to distributed Fresh Air content through limited outlets, like RSS, based on the terms of the contract. At the time of launch, however, NPR was not permitted to offer Fresh Air content through the API using the richer output formats. By the December 2008 upgrade to the API, however, the contract was renegotiated to include distribution through the API. This highlights two points. First, at launch, we needed to incorporate a rights management system in the API that could identify specific types of content and then restrict that content from being distributed for certain types of users. The second key point is that NPR has been shifting our contract strategy to enable more content that we pick up to be distributable anywhere NPR content appears, including through the API. Rights Tagging System Our system for tagging assets not produced by NPR is critical for the success of rights management. That said, a sizable portion of this system involves manual effort. After all, it is the editorial process that chooses stories from external sources (e.g. AP, Reuters, etc.), images, videos and other assets. Upon selection of these assets, editorial staff then enter them into our content management system that contains appropriate fields for tagging the owner of the content. Of course, we do have scripts that pull in some materials, like the AP Business feeds on our site. Those stories and assets that get pulled in through automated systems also get tagged by the scripts. Finally, we also have scripts to remove content from our system based on contractual obligations. For example, if we have the rights to present an image for only 30 days, these scripts will purge the system of that image and its metadata at the appropriate time. Rights Management System After we determine what we are allowed to do based on the contracts, and after appropriately tagging the content itself, we were able to create a pretty flexible and powerful system for managing the distribution of the content through the API. This system has four aspects to it, including query-level filtering, story-level filtering, asset-level filtering and user permissions. Query-level filtering enables the system to remove any story or list (ie. topic, program, series, etc.) from the system due to the permissions. It does this in two ways. First, the system will analyze the API query for any IDs that the user does not have permissions to access. If, for example, the user does not have the rights to view content from This I Believe and the user has included id=4538138 in their API query, the story-level filtering will remove the ID from the query and will proceed to execute the query without it. Once a valid query passes through the system and figures out what stories to return, the story-level filter gets applied. This filter determines which individual stories need to be removed before returning the feed back to the user. This is done by applying the list of IDs in the filter, for the user's access level, as exclusions in the query to the API. The list of IDs in the filter include list IDs (eg. topics, programs, series, etc.), so the same rule applies to any stories that belong to any of these lists. For example, we have already established that my API key does not give me permissions to see stories that belong to This I Believe. If I request the top 10 stories that belong to the Opinion topic, and if the third story is a This I Believe story, then the system will eliminate the the third story and will add the eleventh to the results to accommodate my request for 10 stories. Asset-level filtering is less stringent that story-level filtering in that it does not remove the story completely (as in the example above). Rather, it will display the story, but will only return those assets that the user has the rights to see. For example, if I request the top 10 stories from the People & Places topic, that result set may include a story from Fresh Air and This I Believe. In this case, let's say story number three is still a This I Believe story and story number seven is a Fresh Air story. We have already established that my API key does not allow me to see This I Believe, so the story-level filter will remove the third story and will include the eleventh in my results. Meanwhile, my API key allows me to see Fresh Air stories, just not all of them (any such restriction is no longer the case, but when we first launched the API, Fresh Air was only available through RSS). As a result, the seventh story will get through the story-level filter, but the asset-level filter will remove all assets other than the RSS information. We have other asset-level filters for audio, images, video, full text, etc. The final element of this system, which has been mentioned throughout, is permissions. Our permission levels include Public, Partner, Station, NPR.org and Master, with increasing level of access in that order. For each level, there is a distinct list of IDs associated with each filter type (although the query and story filter lists are always the same). As a result, the same story in our system can theoretically be removed for the Public user, only have RSS content for Partner users, have everything but images for Stations, and be fully available to the NPR.org users. Meanwhile, a different story can theoretically have a completely different permission scheme enabling NPR.org users no access to it while public users can see it all. To see how this filtering layer sits on top of our system, here is an architectural diagram: Click here to enlarge Ongoing Challenges Although this system handles our cases for the most part, rights filtering is and will always be a challenge. There are certainly cases that could sneak through the system. These cases could be a result of the editorial process, the tagging tools or the code in the API. We also encounter new scenarios that sometimes require us to quickly modify the API to handle them. Despite these challenges, we have been pretty happy with this system so far. --Daniel Jacobson One of the things that I am most commonly asked about regarding the NPR API is rights management. Because we are distributing content to unknown destinations, it is critical to make sure the API itself can control what gets offered and to whom. To handle these kinds of issues, we built a robust permissions and rights management system into the API. But that is not enough. Rights management starts with contracts and ensuring that the content is tagged appropriately. Without these steps, the rights management system cannot accurately withhold the content that is not allowed to be distributed. So, here is a breakdown of the steps we went through and the systems we built to handle rights in our API.

Contracts
Before launching the API, we spent a lot of time with our legal team reviewing existing contracts and our rights tagging system. Based on this review, we determined that a few changes needed to be made to the rights tagging system, but there were quite a few restrictions on what could be offered through the API. One interesting example is Fresh Air. Fresh Air is a program produced by WHYY and distributed on the radio by NPR. NPR is also responsible for displaying the content on NPR.org and is allowed to distributed Fresh Air content through limited outlets, like RSS, based on the terms of the contract. At the time of launch, however, NPR was not permitted to offer Fresh Air content through the API using the richer output formats. By the December 2008 upgrade to the API, however, the contract was renegotiated to include distribution through the API.

This highlights two points. First, at launch, we needed to incorporate a rights management system in the API that could identify specific types of content and then restrict that content from being distributed for certain types of users. The second key point is that NPR has been shifting our contract strategy to enable more content that we pick up to be distributable anywhere NPR content appears, including through the API.

Rights Tagging System
Our system for tagging assets not produced by NPR is critical for the success of rights management. That said, a sizable portion of this system involves manual effort. After all, it is the editorial process that chooses stories from external sources (e.g. AP, Reuters, etc.), images, videos and other assets. Upon selection of these assets, editorial staff then enter them into our content management system that contains appropriate fields for tagging the owner of the content.

Of course, we do have scripts that pull in some materials, like the AP Business feeds on our site. Those stories and assets that get pulled in through automated systems also get tagged by the scripts.

Finally, we also have scripts to remove content from our system based on contractual obligations. For example, if we have the rights to present an image for only 30 days, these scripts will purge the system of that image and its metadata at the appropriate time.

Rights Management System
After we determine what we are allowed to do based on the contracts, and after appropriately tagging the content itself, we were able to create a pretty flexible and powerful system for managing the distribution of the content through the API. This system has four aspects to it, including query-level filtering, story-level filtering, asset-level filtering and user permissions.

Query-level filtering enables the system to remove any story or list (ie. topic, program, series, etc.) from the system due to the permissions. It does this in two ways. First, the system will analyze the API query for any IDs that the user does not have permissions to access. If, for example, the user does not have the rights to view content from This I Believe and the user has included id=4538138 in their API query, the story-level filtering will remove the ID from the query and will proceed to execute the query without it.

Once a valid query passes through the system and figures out what stories to return, the story-level filter gets applied. This filter determines which individual stories need to be removed before returning the feed back to the user. This is done by applying the list of IDs in the filter, for the user's access level, as exclusions in the query to the API. The list of IDs in the filter include list IDs (eg. topics, programs, series, etc.), so the same rule applies to any stories that belong to any of these lists. For example, we have already established that my API key does not give me permissions to see stories that belong to This I Believe. If I request the top 10 stories that belong to the Opinion topic, and if the third story is a This I Believe story, then the system will eliminate the the third story and will add the eleventh to the results to accommodate my request for 10 stories.

Asset-level filtering is less stringent that story-level filtering in that it does not remove the story completely (as in the example above). Rather, it will display the story, but will only return those assets that the user has the rights to see. For example, if I request the top 10 stories from the People & Places topic, that result set may include a story from Fresh Air and This I Believe. In this case, let's say story number three is still a This I Believe story and story number seven is a Fresh Air story. We have already established that my API key does not allow me to see This I Believe, so the story-level filter will remove the third story and will include the eleventh in my results. Meanwhile, my API key allows me to see Fresh Air stories, just not all of them (any such restriction is no longer the case, but when we first launched the API, Fresh Air was only available through RSS). As a result, the seventh story will get through the story-level filter, but the asset-level filter will remove all assets other than the RSS information. We have other asset-level filters for audio, images, video, full text, etc.

The final element of this system, which has been mentioned throughout, is permissions. Our permission levels include Public, Partner, Station, NPR.org and Master, with increasing level of access in that order. For each level, there is a distinct list of IDs associated with each filter type (although the query and story filter lists are always the same). As a result, the same story in our system can theoretically be removed for the Public user, only have RSS content for Partner users, have everything but images for Stations, and be fully available to the NPR.org users. Meanwhile, a different story can theoretically have a completely different permission scheme enabling NPR.org users no access to it while public users can see it all.

To see how this filtering layer sits on top of our system, here is an architectural diagram:



Click here to enlarge

Ongoing Challenges
Although this system handles our cases for the most part, rights filtering is and will always be a challenge. There are certainly cases that could sneak through the system. These cases could be a result of the editorial process, the tagging tools or the code in the API. We also encounter new scenarios that sometimes require us to quickly modify the API to handle them. Despite these challenges, we have been pretty happy with this system so far.

--Daniel Jacobson

]]> <p><a href="http://www.npr.org/blogs/inside/2009/06/nprs_api_rights_management.html#email">&raquo; E-Mail This</a>&nbsp;&nbsp;&nbsp;&nbsp; <a href="http://del.icio.us/post?url=http://www.npr.org/blogs/inside/2009/06/nprs_api_rights_management.html">&raquo; Add to Del.icio.us</a> </p> <p> <a rel="nofollow" href="http://u.npr.org/adclick/utype=rss/aamsz=300x80/position=rss1/site=NPR/blog=91000411"> <img border="0" width="300" height="80" src="http://u.npr.org/iserver/utype=rss/aamsz=300x80/position=rss1/site=NPR/blog=91000411" /> </a> </p> http://www.npr.org/blogs/inside/2009/06/nprs_api_rights_management.html?ft=1&f=91000411 http://www.npr.org/blogs/inside/2009/06/nprs_api_rights_management.html?ft=1&f=91000411 API API permissions rights Mon, 08 Jun 2009 09:55:58 -0500 Download Your Favorite NPR Stories Today we added a new feature to npr.org that has been at the top of the request list for many of you -- the ability to download audio for individual stories right from the story page. On almost every story from NPR news and music you now have the ability to take the audio with you and enjoy it anywhere. While online you can download and save your favorite interviews, reports, performances and more. Then add the audio to your mp3 player, phone, laptop or netbook to take with you and listen offline wherever you go. It's one more way NPR is making it easier for you to listen and share. To download audio from NPR, just find any story like this one from Nina Totenberg, Obama Picks Sotomayor For High Court click on the link with the red 'down' arrow and a download of the mp3 file will begin automatically. You will find several types of stories don't have a download available. These will usually be things like exclusive music tracks, live concerts or acquired programs we don't have all the rights to. Not all our archives are available in a format that's downloadable, so you won't find links to those stories either but we'll be adding more in the future. You may also notice these mp3s don't have all the ID3 tags and information about the story you might like to find tagged in the file. This is an improvement we are working on and will be bringing to you soon to make it easier to organize all the NPR stories saved in your collection. We hope you'll find this new feature a quick and easy way to enjoy and share your favorite NPR stories. Whether you're online or off, we want you to always be able to take NPR along with you. Thanks for all the support and ideas, keep letting us know more ways we can help bring NPR to you wherever you are. -- Adam J Martin Today we added a new feature to npr.org that has been at the top of the request list for many of you -- the ability to download audio for individual stories right from the story page.

On almost every story from NPR news and music you now have the ability to take the audio with you and enjoy it anywhere. While online you can download and save your favorite interviews, reports, performances and more. Then add the audio to your mp3 player, phone, laptop or netbook to take with you and listen offline wherever you go. It's one more way NPR is making it easier for you to listen and share.

To download audio from NPR, just find any story like this one from Nina Totenberg, Obama Picks Sotomayor For High Court click on the link with the red 'down' arrow and a download of the mp3 file will begin automatically.

You will find several types of stories don't have a download available. These will usually be things like exclusive music tracks, live concerts or acquired programs we don't have all the rights to. Not all our archives are available in a format that's downloadable, so you won't find links to those stories either but we'll be adding more in the future. You may also notice these mp3s don't have all the ID3 tags and information about the story you might like to find tagged in the file. This is an improvement we are working on and will be bringing to you soon to make it easier to organize all the NPR stories saved in your collection.

We hope you'll find this new feature a quick and easy way to enjoy and share your favorite NPR stories. Whether you're online or off, we want you to always be able to take NPR along with you.

Thanks for all the support and ideas, keep letting us know more ways we can help bring NPR to you wherever you are.
-- Adam J Martin

]]> <p><a href="http://www.npr.org/blogs/inside/2009/05/download_your_favorite_npr_stories.html#email">&raquo; E-Mail This</a>&nbsp;&nbsp;&nbsp;&nbsp; <a href="http://del.icio.us/post?url=http://www.npr.org/blogs/inside/2009/05/download_your_favorite_npr_stories.html">&raquo; Add to Del.icio.us</a> </p> http://www.npr.org/blogs/inside/2009/05/download_your_favorite_npr_stories.html?ft=1&f=91000411 http://www.npr.org/blogs/inside/2009/05/download_your_favorite_npr_stories.html?ft=1&f=91000411 Wed, 27 May 2009 18:26:36 -0500 Experimenting with Facebook Connect Earlier today, we upgraded the software we use on NPR.org for commenting and other community tools. The upgrade addresses a number of bugs in the system, but it also includes our first experiment with Facebook Connect. For those of you who aren't familiar with it, Facebook Connect is a way of letting Facebook users participate in other websites and share their content back to their Facebook accounts. In our first experiment, we've added a "Post this comment to Facebook" option to our comments. When you post a comment on NPR.org and click that option, you'll be able to connect your NPR.org account with your Facebook account and have your NPR.org cross-posted to your Facebook news feed. For now, that's as far as we've taken it, and it's still not perfect - we've gotten it working in Safari and Firefox, but IE users may not be able to see the "Post this comment to Facebook" checkbox just yet. I'll let you know when that's resolved. Meanwhile, exploring the possibility of expanding this feature so you can cross-post NPR stories to Facebook when you click the "recommen" button, and perhaps even use your Facebook login and password to sign into the NPR Community. We're also exploring OpenID, Open Social and other tools that would allow people to sign into NPR via other websites and share their content more easily. Anyway, I hope you find our first experiment with Facebook Connect useful. Please let us know what you think. -- Andy Carvin Earlier today, we upgraded the software we use on NPR.org for commenting and other community tools. The upgrade addresses a number of bugs in the system, but it also includes our first experiment with Facebook Connect.

For those of you who aren't familiar with it, Facebook Connect is a way of letting Facebook users participate in other websites and share their content back to their Facebook accounts. In our first experiment, we've added a "Post this comment to Facebook" option to our comments. When you post a comment on NPR.org and click that option, you'll be able to connect your NPR.org account with your Facebook account and have your NPR.org cross-posted to your Facebook news feed.

For now, that's as far as we've taken it, and it's still not perfect - we've gotten it working in Safari and Firefox, but IE users may not be able to see the "Post this comment to Facebook" checkbox just yet. I'll let you know when that's resolved. Meanwhile, exploring the possibility of expanding this feature so you can cross-post NPR stories to Facebook when you click the "recommen" button, and perhaps even use your Facebook login and password to sign into the NPR Community. We're also exploring OpenID, Open Social and other tools that would allow people to sign into NPR via other websites and share their content more easily.

Anyway, I hope you find our first experiment with Facebook Connect useful. Please let us know what you think.

-- Andy Carvin

]]> <p><a href="http://www.npr.org/blogs/inside/2009/04/experimenting_with_facebook_co.html#email">&raquo; E-Mail This</a>&nbsp;&nbsp;&nbsp;&nbsp; <a href="http://del.icio.us/post?url=http://www.npr.org/blogs/inside/2009/04/experimenting_with_facebook_co.html">&raquo; Add to Del.icio.us</a> </p> http://www.npr.org/blogs/inside/2009/04/experimenting_with_facebook_co.html?ft=1&f=91000411 http://www.npr.org/blogs/inside/2009/04/experimenting_with_facebook_co.html?ft=1&f=91000411 Social Media Facebook Facebook Connect OpenSocial comments openid Wed, 29 Apr 2009 13:32:44 -0500 NPR.org Nominated for 5 Webby, 2 EPpy It has been a couple of rewarding weeks here at NPR.org. We recently received notice that we had five Webby nominations and 2 Webby honoree mentions. Last week it was announced that we are finalist for two EPpy awards including one for the NPR API. And just to round things out, we picked up the Golden Dot Award for Vote Report. The 2009 EPpy Awards are the 14th year of the program honoring the best Web sites in the media world, including newspapers, TV-cable, radio and magazines. The NPR API was nominated in the "Best Community Service Application in a Media-Affiliated Web Site with more than one million unique monthly visitors" category. Additionally, NPR.org was nominated overall for "Best Radio Affiliated Web Site". Other finalist in this category include Minnesota Public Radio and American RadioWorks from American Public Media. The Webby Awards, an international award, was established in 1996 to honor excellence on the Internet. You can vote for the people's choice Webby's here. Our 2009 nominations include: NPR Podcasts in the Podcasts category Project Song in the Music category NPR Music in the Music category NPR.org in the Radio category NPR iPhone Site in the News category We also received Webby "honoree" mentions for: Monitor Mix in the Blog -- Culture/Personal category NPR Music Mobile in the Entertainment category Finally as mentioned, last week we picked up the Golden Dot Award for the twitter Vote Report for Best Animation or Mashup. Woot! Thanks Everyone! -- Zach Brand It has been a couple of rewarding weeks here at NPR.org.

We recently received notice that we had five Webby nominations and 2 Webby honoree mentions. Last week it was announced that we are finalist for two EPpy awards including one for the NPR API. And just to round things out, we picked up the Golden Dot Award for Vote Report.

The 2009 EPpy Awards are the 14th year of the program honoring the best Web sites in the media world, including newspapers, TV-cable, radio and magazines. The NPR API was nominated in the "Best Community Service Application in a Media-Affiliated Web Site with more than one million unique monthly visitors" category.

Additionally, NPR.org was nominated overall for "Best Radio Affiliated Web Site". Other finalist in this category include Minnesota Public Radio and American RadioWorks from American Public Media.

The Webby Awards, an international award, was established in 1996 to honor excellence on the Internet. You can vote for the people's choice Webby's here. Our 2009 nominations include:

    NPR Podcasts in the Podcasts category
    Project Song in the Music category
    NPR Music in the Music category
    NPR.org in the Radio category
    NPR iPhone Site in the News category

We also received Webby "honoree" mentions for:

    Monitor Mix in the Blog -- Culture/Personal category
    NPR Music Mobile in the Entertainment category

Finally as mentioned, last week we picked up the Golden Dot Award for the twitter Vote Report for Best Animation or Mashup.

Woot! Thanks Everyone!

-- Zach Brand

]]> <p><a href="http://www.npr.org/blogs/inside/2009/04/nprorg_nominated_for_5_webby_2.html#email">&raquo; E-Mail This</a>&nbsp;&nbsp;&nbsp;&nbsp; <a href="http://del.icio.us/post?url=http://www.npr.org/blogs/inside/2009/04/nprorg_nominated_for_5_webby_2.html">&raquo; Add to Del.icio.us</a> </p> http://www.npr.org/blogs/inside/2009/04/nprorg_nominated_for_5_webby_2.html?ft=1&f=91000411 http://www.npr.org/blogs/inside/2009/04/nprorg_nominated_for_5_webby_2.html?ft=1&f=91000411 API apps mobile social networking Mon, 27 Apr 2009 15:01:40 -0500 NPR on the iPhone Ever since NPR launched an API last July, a number of developers have rushed to build free applications that make public radio content available on a variety of devices, especially the iPhone. From generic podcast players to the NPR Station Finder, these apps are all competing to offer the best experience for consuming NPR content. These apps all have something else in common: not one of them was built or sponsored by NPR. Instead, they are the work of NPR fans who wanted to make a unique gift to the community of public radio listeners. Like the monetary gifts that thousands of listeners make to their local public radio station each year, these gifts of custom applications help to share the public radio experience with new generations of listeners. One of my favorite applications is NPR Addict, which makes more than 13 years of NPR content available on the iPhone. Developed by Bradley Flubacher, a professional coder who moonlights as a volunteer firefighter, NPR Addict features podcasts and streams from NPR stations across the nation. Flubacher continues to update the app, in spite of his busy schedule, and every few months I notice another feature that keeps me glued to my iPhone for the weekend. To all the Flubachers of the world, we at NPR want to say thank you. Thank you for your time, your innovative spirit, and for sharing our love of public radio. -- Demian Perry Ever since NPR launched an API last July, a number of developers have rushed to build free applications that make public radio content available on a variety of devices, especially the iPhone. From generic podcast players to the NPR Station Finder, these apps are all competing to offer the best experience for consuming NPR content.

These apps all have something else in common: not one of them was built or sponsored by NPR. Instead, they are the work of NPR fans who wanted to make a unique gift to the community of public radio listeners. Like the monetary gifts that thousands of listeners make to their local public radio station each year, these gifts of custom applications help to share the public radio experience with new generations of listeners.

One of my favorite applications is NPR Addict, which makes more than 13 years of NPR content available on the iPhone. Developed by Bradley Flubacher, a professional coder who moonlights as a volunteer firefighter, NPR Addict features podcasts and streams from NPR stations across the nation. Flubacher continues to update the app, in spite of his busy schedule, and every few months I notice another feature that keeps me glued to my iPhone for the weekend.

To all the Flubachers of the world, we at NPR want to say thank you. Thank you for your time, your innovative spirit, and for sharing our love of public radio.
-- Demian Perry

]]> <p><a href="http://www.npr.org/blogs/inside/2009/04/npr_on_the_iphone.html#email">&raquo; E-Mail This</a>&nbsp;&nbsp;&nbsp;&nbsp; <a href="http://del.icio.us/post?url=http://www.npr.org/blogs/inside/2009/04/npr_on_the_iphone.html">&raquo; Add to Del.icio.us</a> </p> http://www.npr.org/blogs/inside/2009/04/npr_on_the_iphone.html?ft=1&f=91000411 http://www.npr.org/blogs/inside/2009/04/npr_on_the_iphone.html?ft=1&f=91000411 Mobile iPhone Wed, 22 Apr 2009 10:20:09 -0500 New API Feature : Create Your Own XML Output Today, we added two updates to the API, as follows: XML Field Remap This new functionality allows you to modify our NPRML elements to whatever you want, so your API requests can fit your existing applications without you having to change your code. The remap function allows any node or any attribute to be renamed and it can apply to any number of elements in the document. And again, this only applies to the NPRML output. To see how it works, go to the API Input Reference. In the meantime, here are some examples of how to modify the API query string to implement the remap: - To change the list element, use "remap=list:newList", which will rename the list node to "newList". - To change a sub-element of list, use "remap="list.title:newListTitle", which will rename the title node under list to "newListTitle". - To change the story element, use "remap=list.story:newStory", which will rename the story node to "newStory". - To change a sub-element of story, use "remap=list.story.title:newStoryTitle", which will rename the title node under story to "newStoryTitle". - To change a attribute for any element in the NPRML output (even if the node itself was changed), use "remap=story~id:newStoryId", which will rename the id attribute for the story node to "newStoryId". - To apply many of these changes in a single query, use the comma to separate the remap commands, as follows: "remap=list:newList,story:newStory,story.teaser:newTeaser, story~id:newStoryId,list.story.text.paragraph:textParagraph". Most Emailed Feed We also opened up the Most Emailed list through the API. Previously, it was only available as an RSS feed, but now, it can be accessed through the API, including access to full text, audio, images, and other assets that NPR has the rights to redistribute. There are a few limitations in the feed, however, that are not present in any of our other existing options in the API. For example, this feed cannot be mashed-up with any other feeds from the API, it cannot be sorted, and the queries cannot be restricted by date or search term. As a result of these limitations, the Most Emailed feed is also not present in the Query Generator. To acces the Most Emailed feed, add "id=100" to the API query string. --Daniel Jacobson Today, we added two updates to the API, as follows:

XML Field Remap
This new functionality allows you to modify our NPRML elements to whatever you want, so your API requests can fit your existing applications without you having to change your code. The remap function allows any node or any attribute to be renamed and it can apply to any number of elements in the document. And again, this only applies to the NPRML output. To see how it works, go to the API Input Reference. In the meantime, here are some examples of how to modify the API query string to implement the remap:

- To change the list element, use "remap=list:newList", which will rename the list node to "newList".

- To change a sub-element of list, use "remap="list.title:newListTitle", which will rename the title node under list to "newListTitle".

- To change the story element, use "remap=list.story:newStory", which will rename the story node to "newStory".

- To change a sub-element of story, use "remap=list.story.title:newStoryTitle", which will rename the title node under story to "newStoryTitle".

- To change a attribute for any element in the NPRML output (even if the node itself was changed), use "remap=story~id:newStoryId", which will rename the id attribute for the story node to "newStoryId".

- To apply many of these changes in a single query, use the comma to separate the remap commands, as follows: "remap=list:newList,story:newStory,story.teaser:newTeaser,
story~id:newStoryId,list.story.text.paragraph:textParagraph".

Most Emailed Feed
We also opened up the Most Emailed list through the API. Previously, it was only available as an RSS feed, but now, it can be accessed through the API, including access to full text, audio, images, and other assets that NPR has the rights to redistribute. There are a few limitations in the feed, however, that are not present in any of our other existing options in the API. For example, this feed cannot be mashed-up with any other feeds from the API, it cannot be sorted, and the queries cannot be restricted by date or search term. As a result of these limitations, the Most Emailed feed is also not present in the Query Generator.

To acces the Most Emailed feed, add "id=100" to the API query string.
--Daniel Jacobson

]]> <p><a href="http://www.npr.org/blogs/inside/2009/04/new_api_feature_create_your_ow.html#email">&raquo; E-Mail This</a>&nbsp;&nbsp;&nbsp;&nbsp; <a href="http://del.icio.us/post?url=http://www.npr.org/blogs/inside/2009/04/new_api_feature_create_your_ow.html">&raquo; Add to Del.icio.us</a> </p> http://www.npr.org/blogs/inside/2009/04/new_api_feature_create_your_ow.html?ft=1&f=91000411 http://www.npr.org/blogs/inside/2009/04/new_api_feature_create_your_ow.html?ft=1&f=91000411 API API NPRML Mon, 06 Apr 2009 10:28:27 -0500 API Update : "Day to Day" and "News & Notes" To Be Retired NPR's programs "Day to Day" and "News & Notes" will be broadcasting their final shows on Friday, March 20, 2009. Although the programs will no longer be producing new shows, the entire archive for both of these shows will still be available on NPR.org and through the API. Eventually, we will likely remove these programs from the API Query Generator, although their IDs will still be valid in the API and can be found in the API Mapping Index. API queries that use these program IDs will not return any new content after this Friday. The IDs, however, will remain valid, so your applications should continue to work as expected. Finally, to access the full archives of these programs in the API, you can use the functions available in the Control tab of the Query Generator. These functions include searches based on search terms and date ranges and allow for the ability to paginate through the results. Please let us know if anything unexpected happens as a result of this change. --Daniel Jacobson NPR's programs "Day to Day" and "News & Notes" will be broadcasting their final shows on Friday, March 20, 2009. Although the programs will no longer be producing new shows, the entire archive for both of these shows will still be available on NPR.org and through the API. Eventually, we will likely remove these programs from the API Query Generator, although their IDs will still be valid in the API and can be found in the API Mapping Index.

API queries that use these program IDs will not return any new content after this Friday. The IDs, however, will remain valid, so your applications should continue to work as expected.

Finally, to access the full archives of these programs in the API, you can use the functions available in the Control tab of the Query Generator. These functions include searches based on search terms and date ranges and allow for the ability to paginate through the results.

Please let us know if anything unexpected happens as a result of this change.
--Daniel Jacobson

]]> <p><a href="http://www.npr.org/blogs/inside/2009/03/api_update_day_to_day_and_news.html#email">&raquo; E-Mail This</a>&nbsp;&nbsp;&nbsp;&nbsp; <a href="http://del.icio.us/post?url=http://www.npr.org/blogs/inside/2009/03/api_update_day_to_day_and_news.html">&raquo; Add to Del.icio.us</a> </p> <p> <a rel="nofollow" href="http://u.npr.org/adclick/utype=rss/aamsz=300x80/position=rss2/site=NPR/blog=91000411"> <img border="0" width="300" height="80" src="http://u.npr.org/iserver/utype=rss/aamsz=300x80/position=rss2/site=NPR/blog=91000411" /> </a> </p> http://www.npr.org/blogs/inside/2009/03/api_update_day_to_day_and_news.html?ft=1&f=91000411 http://www.npr.org/blogs/inside/2009/03/api_update_day_to_day_and_news.html?ft=1&f=91000411 API API Thu, 19 Mar 2009 15:26:32 -0500 Seeking Feedback for SXSW API Session As mentioned in Zach's previous post, I will be part of a panel at SXSW. The panel discussion will be on APIs, is called "Get Me Rewrite! Developing APIs and the Changing Face of News", and is on Sunday at 3:30pm. For more information on the panel, go to the SXSW page for this panel. The panel moderator is Jacob Harris, from The New York Times. Joining the discussion will be Brad Stenger from Wired, and John Donovan from Daylife. We will have a substantial time set aside for Q&A although prior to the Q&A we will be addressing many of the challenges in producing and maintaining APIs. That said, there are myriad things we can focus on when discussing APIs... So, please let us know what is most on your mind. What kinds of questions do you want this panel to answer? Are you interested in technical background, business goals, legal issues, getting corporate buy-in, the marketplace for APIs, etc.? We will be using this feedback to refine our topics accordingly as we finish preparing for the session. --Daniel Jacobson As mentioned in Zach's previous post, I will be part of a panel at SXSW. The panel discussion will be on APIs, is called "Get Me Rewrite! Developing APIs and the Changing Face of News", and is on Sunday at 3:30pm. For more information on the panel, go to the SXSW page for this panel.

The panel moderator is Jacob Harris, from The New York Times. Joining the discussion will be Brad Stenger from Wired, and John Donovan from Daylife.

We will have a substantial time set aside for Q&A although prior to the Q&A we will be addressing many of the challenges in producing and maintaining APIs. That said, there are myriad things we can focus on when discussing APIs...

So, please let us know what is most on your mind. What kinds of questions do you want this panel to answer? Are you interested in technical background, business goals, legal issues, getting corporate buy-in, the marketplace for APIs, etc.? We will be using this feedback to refine our topics accordingly as we finish preparing for the session.

--Daniel Jacobson

]]> <p><a href="http://www.npr.org/blogs/inside/2009/03/seeking_feedback_for_our_sxsw.html#email">&raquo; E-Mail This</a>&nbsp;&nbsp;&nbsp;&nbsp; <a href="http://del.icio.us/post?url=http://www.npr.org/blogs/inside/2009/03/seeking_feedback_for_our_sxsw.html">&raquo; Add to Del.icio.us</a> </p> http://www.npr.org/blogs/inside/2009/03/seeking_feedback_for_our_sxsw.html?ft=1&f=91000411 http://www.npr.org/blogs/inside/2009/03/seeking_feedback_for_our_sxsw.html?ft=1&f=91000411 API API SXSW feedback Wed, 11 Mar 2009 09:46:00 -0500 NPR Roadshow cont'd As follow up from my post back in November there are a number of events upcoming to hear and meet with folks from NPR Digital Media first hand. Feb. 19-21 - IMA Public Media 09 For those in Public Broadcasting family we hope to see you next week at IMA Public Media 09 . It is scheduled to be: five idea filled days with 55 sessions on everything from coding widgets and measuring impact to mastering the subtle techniques of building online communities. As part of the the WebTech Summit on Wednesday 2/19 I will be appearing in a panel to discuss Mobile site development. The Mobile Site Development session will be a panel that includes myself, - Ann Breckbill and Melinda Driscoll, American Public Media; Keith Hopper, NPR/Public Interactive; Matt MacDonald, PRX; and the inimitable Doc Searls. Later on Wednesday I will be discussing the development and use of Widgets and APIs. John Tynan and I will join Andrew Kuklewicz on such topics as how do you develop widgets and how and when should you pull data from other site's APIs or place widgets on your own sites. Also on Wednesday 2/19 Dick Meyer Editorial Director of NPR Digital Media will be discussing the sustainability of commercial sites as part for the Executive Education & management Training program. As part of the core conference our Social Media Guru Andy Carvin will participate in several presentations. On Thursday 2/19 he will be on a panel discussing "A Social Media How To: Choosing and Using the Right Tools. On Friday 2/20 you will have a chance to meet our new boss here at NPR. CEO Vivian Schiller will be addressing the Conference at 2pm. Also on Friday Andy will be on a panel discussing "Social Media: What's Worked and Lessons Learned." Upcoming: On Wednesday 2/26 I will be presenting NPR's revolutionary work in digital media as part of the We Media game changers conference. On March 15th Dan Jacobson will be part of the discussion at SXSW Interactive festival discussing APIs and the changing face of news online. Finally on April 3rd I will be at the O'Reilly Web 2.0 Expo and joining Robin Sloan the VP of strategy at Current, a media company co-founded by Al Gore and Joel Hyatt. We'll be discussing the future of content distribution from both TV and Radio perspectives. We hope to see you soon! -- Zach Brand

As follow up from my post back in November there are a number of events upcoming to hear and meet with folks from NPR Digital Media first hand.

Feb. 19-21 - IMA Public Media 09

For those in Public Broadcasting family we hope to see you next week at IMA Public Media 09 . It is scheduled to be: five idea filled days with 55 sessions on everything from coding widgets and measuring impact to mastering the subtle techniques of building online communities.

  • As part of the the WebTech Summit on Wednesday 2/19 I will be appearing in a panel to discuss Mobile site development. The Mobile Site Development session will be a panel that includes myself, - Ann Breckbill and Melinda Driscoll, American Public Media; Keith Hopper, NPR/Public Interactive; Matt MacDonald, PRX; and the inimitable Doc Searls.

  • Later on Wednesday I will be discussing the development and use of Widgets and APIs. John Tynan and I will join Andrew Kuklewicz on such topics as how do you develop widgets and how and when should you pull data from other site's APIs or place widgets on your own sites.

  • As part of the core conference our Social Media Guru Andy Carvin will participate in several presentations. On Thursday 2/19 he will be on a panel discussing "A Social Media How To: Choosing and Using the Right Tools.

  • On Friday 2/20 you will have a chance to meet our new boss here at NPR. CEO Vivian Schiller will be addressing the Conference at 2pm. Also on Friday Andy will be on a panel discussing "Social Media: What's Worked and Lessons Learned."

Upcoming:

On Wednesday 2/26 I will be presenting NPR's revolutionary work in digital media as part of the We Media game changers conference.

On March 15th Dan Jacobson will be part of the discussion at SXSW Interactive festival discussing APIs and the changing face of news online.

Finally on April 3rd I will be at the O'Reilly Web 2.0 Expo and joining Robin Sloan the VP of strategy at Current, a media company co-founded by Al Gore and Joel Hyatt. We'll be discussing the future of content distribution from both TV and Radio perspectives.

We hope to see you soon!

-- Zach Brand

]]> <p><a href="http://www.npr.org/blogs/inside/2009/02/npr_roadshow_contd.html#email">&raquo; E-Mail This</a>&nbsp;&nbsp;&nbsp;&nbsp; <a href="http://del.icio.us/post?url=http://www.npr.org/blogs/inside/2009/02/npr_roadshow_contd.html">&raquo; Add to Del.icio.us</a> </p> http://www.npr.org/blogs/inside/2009/02/npr_roadshow_contd.html?ft=1&f=91000411 http://www.npr.org/blogs/inside/2009/02/npr_roadshow_contd.html?ft=1&f=91000411 Administrative Stuff API mobile social networking Fri, 13 Feb 2009 11:34:44 -0500 HTML Addressing and Content Portability My last post highlighted the reasons why it is important to maintain clean content. This post will focus on what we call "HTML Addressing", which is a way for us to control the HTML (and other markup) that appears in our text fields, making the content more portable. HTML Addressing has several components to it, some of which are very typical of content management systems, while others are not. These components are described below.

Keep Content Separated into Distinct Fields
It is very important for us to make sure all content is populated into very distinct fields in the database. Some platforms, like blogging systems, have a big text block where images and other asset references are embedded within the text. This not only restricts our ability to modify the display of the image relative to the text, it also prevents us from doing something distinct with the image itself because it is now tightly bound to the text.

Limit HTML to be Allowed in Specific Fields
There are a range for free-text fields in our CMS. Most of these fields do not require any inline markup because the templates take care of all of the display rules. Others, like the teaser and full story text, however, do have instances where that markup does add to the editorial meaning of the content. As a result, we limited the fields that allow markup to only those that actually need them. To enforce the above limitation, we developed a series of JavaScript functions that apply to each field in the CMS to prevent markup from being entered into fields that do not allow HTML.

Limit HTML Tags in Allowable Fields
For those fields that allow HTML, we only allow very specifc tags (and different fields can allow for different tags). For example, in some fields, we may allow tags such as <strong> and <em>, but not <b>, <div> or <img> (<b> is deprecated, <div> could introduce too many variables within the context of the pages, and <img> is not allowed because images should be added in their appropriate fields). Finally, some allowable tags, such as <a>, allow parameters to be applied to them although we do restrict some of the event-based and style-based parameters. Again, our JavaScript functions ensure that only allowable tags are entered into the appropriate fields, that the parameters applied to those tags are viable, and that all tags are closed and nested correctly.

Storage of HTML in Database
Upon saving a story in the CMS, the server-side code identifies all markup in the content and pulls all tags out. In pulling the tags out, we capture the "address" of the open and close tags. By "address", we mean the character position in the field in which the tag appears. For example, in the string "this is a <strong>string</strong> of content", the open <strong> tag starts at character eleven and the close starts at character 25. So, when saving, we store only "this is a string of content" in the database field, but also put into a relational table the necessary information to reconstruct it with the tags. Included in that information are the story's unique ID, the unique ID for the field in the database where the tag was found, the character position for the opening and closing tags (stored as separate records in the database), the unique ID for the tag and any parameter and values attached the tag. When I say that we store the unique ID for the tag, it is because we don't actually store the tag in this table (for reasons I will describe below in the Benefits section).

Other Characters
HTML Addressing generally refers to the handling of HTML markup in our content. The functions, however, do more than just HTML. There are a range of characters that also create problems throughout the system, including smart quotes and mdashes. The HTML Addressing functionality does not store the locations of these characters (which are typically added to the content by copying/pasting from Microsoft Word). Rather, upon saving to the database, it replaces them with comparable characters that are more standard. For example, the smart quotes become regular quotes. This list of replacement characters is extensible.

Apply HTML Addressing to the Archive
The functionality described above applies to new stories getting created in our CMS (or old stories that get resaved). Because our older stories contained a wide array of tags, we also needed to be able to run this functionality against our archive. In doing so, we found tags like <font> that needed treatment. Generally speaking, the rules for the script were to move any tags recognized as valid for the sytem into the model described above while all other tags were removed completely. There were some exceptions to this, but that is generally what happened.

Benefits to HTML Addressing
Because we store the content without the tags, we can then present the content with or without them, depending on the output destination. If the content is getting output to NPR.org, we then recompile the markup into the content based on that relational table. If the content is getting output to podcasts, however, we simply print the raw content without the markup. In the NPR API, you can see an example of the difference between the two in our NPRML format (see the <text> and <textWithHtml> elements).

Obviously, removing the HTML from the content enables our content to be portable and to be distributed to virtually any platform. But similar goals could be met by storing the tags with the content in the database and running stripping scripts against the data when the output destination requires no HTML. So why go through the trouble of stripping out the HTML when storing it?

For us, the primary differentiating factor in stripping them out upon storage instead of upon presentation is in tag management. If tags need to change (for example, the <b> tag was deprecated in favor of <strong>), we need to make that change to only one field in the entire database. If the tags are in the content, we would need to run scripts that cycle through all fields and all records to make the change. Additionally, if we introduce micro-formats or other markup, we can be sure that they are all handled the same way.

Another interesting benefit to this approach is that when we output the content to different platforms, we could actually transform the tags on the fly. For example, since an iPod cannot parse <em> tags, we could print single-quotes in their place for that particular output destination. As different platforms develop their own markup for presentation, we simply need to maintain mappings between HTML and the other presentation markup tags.

Detriments to HTML Addressing
The main problem with HTML Addressing is the fact that the default action for building pages on NPR.org (currently, the primary destination for the content) requires the toughest action. That is, when we render pages for NPR.org, we have to do all the processing to add the HTML back into the content. We mitigate this by burning the content into our XML repository and serving the pages to users through our caching layers. The XML repository contains the compiled output for both the marked-up and unmarked-up content, so when the templates render the output, it simply needs to pull from the appropriate fields. Moreover, the caching layer ensures that the performance is optimized regardless of the output format.

There are obviously many different ways to skin this cat, but this approach has provided us with cleaner and more portable content. This is not to suggest that our system's content is flawless in its portability. There are more challenges than just markup in content that make portability difficult, many of which we deal with on a daily basis. But eliminating markup from the content is a big step in achieving the goal.
--Daniel Jacobson

]]> <p><a href="http://www.npr.org/blogs/inside/2009/02/html_addressing_and_content_po.html#email">&raquo; E-Mail This</a>&nbsp;&nbsp;&nbsp;&nbsp; <a href="http://del.icio.us/post?url=http://www.npr.org/blogs/inside/2009/02/html_addressing_and_content_po.html">&raquo; Add to Del.icio.us</a> </p> http://www.npr.org/blogs/inside/2009/02/html_addressing_and_content_po.html?ft=1&f=91000411 http://www.npr.org/blogs/inside/2009/02/html_addressing_and_content_po.html?ft=1&f=91000411 Technology API Clean Content HTML Portable Content Mon, 09 Feb 2009 13:26:57 -0500 Clean Content = Portable Content As discussed in my earlier post about the strategy of building the API, one of the most important things for content producers is to remain relevant to their users. With content becoming more readily available to these users through distribution channels, it is up to these content producers to make sure the content is where the users are. That does not diminish the need for continued development and maintenance of the Web site, it just means that it is equally important to distribute the content to other places. In order for that to happen, the content has to be portable. So, what does it mean for content to be portable? For NPR Digital Media, one of the primary philosophies driving our systems, is Create Once, Publish Everywhere (COPE). To achieve COPE, here are some key principles that we adhere to: Develop Content Management Tools, not Web Publishing Tools Most content management systems for the online world are used to create Web pages. That said, the Web page is just one possible output for the content (albeit, an important one). In building our CMS at NPR, our goal was to make sure the tool could publish to anything, including NPR.org. If our focus did not consider other platforms, we could have ended up with a Web publishing system that binds the content too closely to the Web site itself. Separate Content from Display As mentioned above, if the content is too closely tied to a specific display, it cannot easily be pushed out to other platforms. Good separation, in addition to facilitating content portability, also makes redesigns of the Web site or alternate presentations of the content of the site easier. For example, because our content is separate from our display, we were able to to launch the NPR Music site without refactoring the system architecture or our presentation layer code. Eliminate markup from content Because we do not know where the content will ultimately end up, it is important to not have platform-specific markup embedded in the content. For example, iPods cannot parse HTML, so we need to make sure our content gets distributed to iPods without tags in it, while the same content must contain the tags for NPR.org. Like many other content management systems, ours captures and stores content in a central database that is completely independent from any presentation layer (I discuss our architecture in this earlier post). For the content to really be portable, however, it needs to work on any platform, including browsers, RSS readers, iPods, radio displays, mobile devices, TVs, etc., which means we must eliminate markup from content, as described above. To solve this problem, we introduced some functionality to our system that we call "HTML Addressing", which will be the topic of my next post. --Daniel Jacobson As discussed in my earlier post about the strategy of building the API, one of the most important things for content producers is to remain relevant to their users. With content becoming more readily available to these users through distribution channels, it is up to these content producers to make sure the content is where the users are. That does not diminish the need for continued development and maintenance of the Web site, it just means that it is equally important to distribute the content to other places. In order for that to happen, the content has to be portable. So, what does it mean for content to be portable? For NPR Digital Media, one of the primary philosophies driving our systems, is Create Once, Publish Everywhere (COPE). To achieve COPE, here are some key principles that we adhere to:

Develop Content Management Tools, not Web Publishing Tools
Most content management systems for the online world are used to create Web pages. That said, the Web page is just one possible output for the content (albeit, an important one). In building our CMS at NPR, our goal was to make sure the tool could publish to anything, including NPR.org. If our focus did not consider other platforms, we could have ended up with a Web publishing system that binds the content too closely to the Web site itself.

Separate Content from Display
As mentioned above, if the content is too closely tied to a specific display, it cannot easily be pushed out to other platforms. Good separation, in addition to facilitating content portability, also makes redesigns of the Web site or alternate presentations of the content of the site easier. For example, because our content is separate from our display, we were able to to launch the NPR Music site without refactoring the system architecture or our presentation layer code.

Eliminate markup from content
Because we do not know where the content will ultimately end up, it is important to not have platform-specific markup embedded in the content. For example, iPods cannot parse HTML, so we need to make sure our content gets distributed to iPods without tags in it, while the same content must contain the tags for NPR.org.

Like many other content management systems, ours captures and stores content in a central database that is completely independent from any presentation layer (I discuss our architecture in this earlier post). For the content to really be portable, however, it needs to work on any platform, including browsers, RSS readers, iPods, radio displays, mobile devices, TVs, etc., which means we must eliminate markup from content, as described above. To solve this problem, we introduced some functionality to our system that we call "HTML Addressing", which will be the topic of my next post.
--Daniel Jacobson

]]> <p><a href="http://www.npr.org/blogs/inside/2009/02/clean_content_portable_content.html#email">&raquo; E-Mail This</a>&nbsp;&nbsp;&nbsp;&nbsp; <a href="http://del.icio.us/post?url=http://www.npr.org/blogs/inside/2009/02/clean_content_portable_content.html">&raquo; Add to Del.icio.us</a> </p> http://www.npr.org/blogs/inside/2009/02/clean_content_portable_content.html?ft=1&f=91000411 http://www.npr.org/blogs/inside/2009/02/clean_content_portable_content.html?ft=1&f=91000411 Technology API Wed, 04 Feb 2009 11:10:03 -0500 Inauguration Report is Live! In case you haven't seen my tweets about it yet, Inauguration Report is now live. We've managed to create a variety of ways for you to share your inauguration experiences, from text messaging to an iPhone app. CBS News and American University are also helping us promote the project. We've created a couple of Web pages you'll want to check out. First, visit NPR's Inauguration Report hub for details on how to participate; there's also a widget there, displaying reports as they come in to us. You can also check out InaugurationReport.com, which displays a giant map of all the reports that have been geotagged. If you're coming to the inauguration or will be involved in events in your community, feel free to start posting dispatches now. We've already gotten hundreds of submissions via Twitter, and other content is coming in as well. We really want to hear from you if you're making your way to DC, whether it's the joy of the road trip or the frustration of traffic gridlock. And on January 20th, we hope to get a ton of submissions, assuming the networks don't come crashing down from the strain. Special thanks to Dave Troy, Andrew Turner. Nathan Freitas and Sze Wong for their spectacular coding work; David Johnson and Dan Farber for joining us in the editorial collaboration; and Nancy Scola and Allison Fine for taking the lead in pulling together the Vote Report team, which directly lead to the creation of this project. We couldn't have done it without you. -- Andy Carvin In case you haven't seen my tweets about it yet, Inauguration Report is now live. We've managed to create a variety of ways for you to share your inauguration experiences, from text messaging to an iPhone app. CBS News and American University are also helping us promote the project.

We've created a couple of Web pages you'll want to check out. First, visit NPR's Inauguration Report hub for details on how to participate; there's also a widget there, displaying reports as they come in to us. You can also check out InaugurationReport.com, which displays a giant map of all the reports that have been geotagged.

If you're coming to the inauguration or will be involved in events in your community, feel free to start posting dispatches now. We've already gotten hundreds of submissions via Twitter, and other content is coming in as well. We really want to hear from you if you're making your way to DC, whether it's the joy of the road trip or the frustration of traffic gridlock. And on January 20th, we hope to get a ton of submissions, assuming the networks don't come crashing down from the strain.

Special thanks to Dave Troy, Andrew Turner. Nathan Freitas and Sze Wong for their spectacular coding work; David Johnson and Dan Farber for joining us in the editorial collaboration; and Nancy Scola and Allison Fine for taking the lead in pulling together the Vote Report team, which directly lead to the creation of this project. We couldn't have done it without you.

-- Andy Carvin

]]> <p><a href="http://www.npr.org/blogs/inside/2009/01/inauguration_report_is_live.html#email">&raquo; E-Mail This</a>&nbsp;&nbsp;&nbsp;&nbsp; <a href="http://del.icio.us/post?url=http://www.npr.org/blogs/inside/2009/01/inauguration_report_is_live.html">&raquo; Add to Del.icio.us</a> </p> http://www.npr.org/blogs/inside/2009/01/inauguration_report_is_live.html?ft=1&f=91000411 http://www.npr.org/blogs/inside/2009/01/inauguration_report_is_live.html?ft=1&f=91000411 Editorial Mobile Social Media Inauguration Day Inauguration Report inaug09 user generated content Fri, 16 Jan 2009 08:45:37 -0500 The Guts of the Station Finder Map As my previous post mentioned, we recently re-launched our Station Finder Map. This post will discuss in more detail how the map works. Now, to the guts... The Underlying Data The system has several underlying database tables, including zip codes, cities and station data. The zip code and city tables, in addition to containing information about the locations, also include the latitude and longitude for the centroid each location. These are pretty simple, flat tables that contain the approximately 41,000 zip codes and 150,000 cities in the United States. The station tables, on the other hand, are much more complex. They contain all of the nearly 2000 stations that carry NPR programming (as well as their translators) along with a wide array of information about those stations, including licensee data and pertinent URLs associated with the station (e.g. their home page, schedule page, donation page, audio streams, RSS feeds and podcast feeds). These tables also include the latitude, longitude and broadcast power information for each antenna. The broadcast power information tells us how far that antenna's broadcast signal can reach in each direction. Our data is broken up into 72 directions, starting with due north and shifting five degrees around the circle until we are back at due north. For each direction, our database contains five different ranges, detailing how far the antenna can reach in that given direction. The range itself determines two things. First, it tells us how far away you can be from the antenna and still hear its signal - this takes into account some impediments, such as mountains. The second thing it tells us is what the quality of the signal will be. The closer you are to the antenna, generally, the more clear the signal will be (although this is not always the case). Finally, most of the data in these tables is publicly available in our recently launched Station Finder API (the coverage data is not available, but everything else is). The functionality of the map is driven off of the API. How Does the Search Work? At the core, the system works based on latitudes and longitudes. If you search the system by zip code or city/state, the system will convert the search term into a latitude and longitude before looking for stations. Similarly, when you look for NPR stations along a driving route, the system identifies a series of points along the route and converts those points into latitudes and longitudes. The waypoints for driving routes include any turn, crossing of a border, start and end points, and some artificially inserted points that we create. (Searches based on call letters bypass the geo-searches and hits the station tables directly.) Once we have the latitude and longitude, we perform a series of calculations based on the Great Circle Calcuation (GCC), which helps us to determine distances on a curved surface (ie. the Earth - and we are assuming that it is not flat). Using the GCC, we look for stations near the latitude and longitude, based on a 100 mile radius from that point. From that list of stations, which is too inclusive, we start our process of narrowing down the results to the actual stations that can be heard. For each station returned from our initial search, we first determine the direction (one of the 72 described earlier) from the antenna to the requested latitude/longitude. Then we find out the distance between the antenna and the latitude/longitude using the GCC. Once we have the distance and the direction, we simply need to do a lookup in our database to determine if the broadcast distance of the station is greater than the distance between the antenna and the latitude/longitude. If the broadcast distance is greater, then the station can be heard in the latitude/longitude. If it is not, then the station cannot be heard. Now, when I say "check to see if the broadcast distance is greater", we are really checking five different broadcast distances in the database. We do this to find out what the quality of the broadcast signal will be for that latitude/longitude. The further the distance, assuming it is still within range, the more likely the signal will worsen. There are other variables, but that is the basic idea. Displaying on the Map The display of this information on the map is pretty straight-forward. We simply drop an antenna icon at each latitude/longitude where a station's antenna is actually located. For that antenna, we use the polygon feature in Virtual Earth to draw and shade the coverage circles on the map. The contours of the coverage circles are drawn by taking the distance of the broadcast range in each of the 72 directions, drawing a line connecting the points, then shading in the circle. We do this for three of the five broadcast ranges in our database. The overlay of the shading for each of these three circles results in the inner circle being darker than the middle circle, which is darker than the outer circle. Other Notes One other thing I should point out about this data is that it is great for the purposes of this type of application - a web-based service to inform our audience as to which NPR stations are available throughout the country. There are other more sophisticated, more precise ways to identify the station coverage maps which are really overkill for this type of service. To see another representation of this same functionality, go to nprroadtrip.com. This is a map mashup produced by an NPR enthusiast (not affiliated with NPR). --Daniel Jacobson As my previous post mentioned, we recently re-launched our Station Finder Map. This post will discuss in more detail how the map works. Now, to the guts...

The Underlying Data
The system has several underlying database tables, including zip codes, cities and station data. The zip code and city tables, in addition to containing information about the locations, also include the latitude and longitude for the centroid each location. These are pretty simple, flat tables that contain the approximately 41,000 zip codes and 150,000 cities in the United States.

The station tables, on the other hand, are much more complex. They contain all of the nearly 2000 stations that carry NPR programming (as well as their translators) along with a wide array of information about those stations, including licensee data and pertinent URLs associated with the station (e.g. their home page, schedule page, donation page, audio streams, RSS feeds and podcast feeds). These tables also include the latitude, longitude and broadcast power information for each antenna.

The broadcast power information tells us how far that antenna's broadcast signal can reach in each direction. Our data is broken up into 72 directions, starting with due north and shifting five degrees around the circle until we are back at due north. For each direction, our database contains five different ranges, detailing how far the antenna can reach in that given direction. The range itself determines two things. First, it tells us how far away you can be from the antenna and still hear its signal - this takes into account some impediments, such as mountains. The second thing it tells us is what the quality of the signal will be. The closer you are to the antenna, generally, the more clear the signal will be (although this is not always the case).

Finally, most of the data in these tables is publicly available in our recently launched Station Finder API (the coverage data is not available, but everything else is). The functionality of the map is driven off of the API.

How Does the Search Work?
At the core, the system works based on latitudes and longitudes. If you search the system by zip code or city/state, the system will convert the search term into a latitude and longitude before looking for stations. Similarly, when you look for NPR stations along a driving route, the system identifies a series of points along the route and converts those points into latitudes and longitudes. The waypoints for driving routes include any turn, crossing of a border, start and end points, and some artificially inserted points that we create. (Searches based on call letters bypass the geo-searches and hits the station tables directly.)

Once we have the latitude and longitude, we perform a series of calculations based on the Great Circle Calcuation (GCC), which helps us to determine distances on a curved surface (ie. the Earth - and we are assuming that it is not flat). Using the GCC, we look for stations near the latitude and longitude, based on a 100 mile radius from that point. From that list of stations, which is too inclusive, we start our process of narrowing down the results to the actual stations that can be heard.

For each station returned from our initial search, we first determine the direction (one of the 72 described earlier) from the antenna to the requested latitude/longitude. Then we find out the distance between the antenna and the latitude/longitude using the GCC. Once we have the distance and the direction, we simply need to do a lookup in our database to determine if the broadcast distance of the station is greater than the distance between the antenna and the latitude/longitude. If the broadcast distance is greater, then the station can be heard in the latitude/longitude. If it is not, then the station cannot be heard.

Now, when I say "check to see if the broadcast distance is greater", we are really checking five different broadcast distances in the database. We do this to find out what the quality of the broadcast signal will be for that latitude/longitude. The further the distance, assuming it is still within range, the more likely the signal will worsen. There are other variables, but that is the basic idea.

Displaying on the Map
The display of this information on the map is pretty straight-forward. We simply drop an antenna icon at each latitude/longitude where a station's antenna is actually located. For that antenna, we use the polygon feature in Virtual Earth to draw and shade the coverage circles on the map. The contours of the coverage circles are drawn by taking the distance of the broadcast range in each of the 72 directions, drawing a line connecting the points, then shading in the circle. We do this for three of the five broadcast ranges in our database. The overlay of the shading for each of these three circles results in the inner circle being darker than the middle circle, which is darker than the outer circle.

Other Notes
One other thing I should point out about this data is that it is great for the purposes of this type of application - a web-based service to inform our audience as to which NPR stations are available throughout the country. There are other more sophisticated, more precise ways to identify the station coverage maps which are really overkill for this type of service.

To see another representation of this same functionality, go to nprroadtrip.com. This is a map mashup produced by an NPR enthusiast (not affiliated with NPR).
--Daniel Jacobson

]]> <p><a href="http://www.npr.org/blogs/inside/2009/01/the_guts_of_the_station_finder.html#email">&raquo; E-Mail This</a>&nbsp;&nbsp;&nbsp;&nbsp; <a href="http://del.icio.us/post?url=http://www.npr.org/blogs/inside/2009/01/the_guts_of_the_station_finder.html">&raquo; Add to Del.icio.us</a> </p> <p> <a rel="nofollow" href="http://u.npr.org/adclick/utype=rss/aamsz=300x80/position=rss3/site=NPR/blog=91000411"> <img border="0" width="300" height="80" src="http://u.npr.org/iserver/utype=rss/aamsz=300x80/position=rss3/site=NPR/blog=91000411" /> </a> </p> http://www.npr.org/blogs/inside/2009/01/the_guts_of_the_station_finder.html?ft=1&f=91000411 http://www.npr.org/blogs/inside/2009/01/the_guts_of_the_station_finder.html?ft=1&f=91000411 Technology Map Station Finder Mon, 12 Jan 2009 15:57:42 -0500