Easily Download any Feed (RSS or Atom) Data in C#

Easily Download any Feed (RSS or Atom) Data in C#
2 votes, 5.00 avg. rating (96% score)

Introduction

Simple and very useful code snippet to download rss feed data to xml file. It works for both RSS and ATOM feeds. It is very useful snippet without putting much lines of code. No need to compare feed types also while downloading

What are covered in this snippet

  • Download feed data irrespective of type rss or atom
  • You will not get error: ‘Element’ is an invalid XmlNodeType
  • Sorting with pubdate within xDocument object
  • Finding the type of feed

Download any Feed Data


	string _masterFile = HostingEnvironment.MapPath("~/feedData.xml");
    string url = "http://en.espnf1.com/rss/motorsport/story/feeds/0.xml?type=2";
	XDocument feedDataObject = null;
	try
	{
		using (WebClient client = new WebClient())
		{
			using (var stream = client.OpenRead(url))
			{
				feedDataObject = XDocument.Load(XmlReader.Create(stream));
			}
		}
	}
	catch (Exception ex)
	{
		Trace.Warn("Feed", "download error: " + url, ex);
	   
	}
   
	//sorting based on pubdate
	var queryItems1 = from c in feedDataObject.Descendants("item") orderby 
						DateTime.Parse(c.Element("pubDate").Value) descending select c;

 

Reading each element

Above snippet is the core, below code is how to read each element. It also reads encoded element value. You can use this snippet to read any wordpress rss easily.
I have added other utility methods

How to detect feed is rss or atom

How to read each elment of the feed (atom and rss)

	string _masterFile = HostingEnvironment.MapPath("~/feedData.xml");
    string url = "http://en.espnf1.com/rss/motorsport/story/feeds/0.xml?type=2"; //espn news feed
    XNamespace content = "http://purl.org/rss/1.0/modules/content/";
	XDocument feedDataObject = null;
	try
	{
		using (WebClient client = new WebClient())
		{
			using (var stream = client.OpenRead(url))
			{
				feedDataObject = XDocument.Load(XmlReader.Create(stream));
			}
		}
		if(feedDataObject != null)
		{
			var query = (from feed in feedDataObject.Descendants("item")
						 orderby DateTime.Parse(feed.Element("pubDate").Value) descending
						 select new
						 {
							 Title		 = feed.Element("title").Value,
							 Description = HttpUtility.HtmlDecode(feed.Element("description").Value),
							 Content     = HttpUtility.HtmlDecode(feed.Element(content.GetName("encoded")).Value),
							 Link        = feed.Element("link").Value,
							 Date        = DateTime.Parse(feed.Element("pubDate").Value).ToShortDateString()
						 });
		}
	}
	catch (Exception ex)
	{
		Trace.Warn("Feed", "download error: " + url, ex);
	   
	}

	public enum FeedType { RSS, ATOM, RDF, INVALID };
	public FeedType GetFeedType(XDocument feedDataObject)
	{
	   int RdfCount = feedDataObject.Root.Descendants().Where(i => i.Name.LocalName == "item").ToList().Count;
	   int AtomCount = feedDataObject.Root.Elements().Where(i => i.Name.LocalName == "entry").ToList().Count;
	   var RssCount = feedDataObject.Root.Descendants().Where(i => i.Name.LocalName == "channel").Elements().Where(i => i.Name.LocalName == "item").ToList().Count;
	   if(RssCount != 0)
	   {
		   return FeedType.RSS;
	   }
	   else if (AtomCount != 0)
	   {
		   return FeedType.ATOM;
	   }
	   else if (RdfCount != 0)
	   {
		   return FeedType.RDF;
	   }
	   else
	   {
		   return FeedType.INVALID;
	   }

	}

	publc class FeedItem
	{
		public string Title { get; set; }
        public string Description { get; set; }
        public string Link { get; set; }
		public DateTime PublishDate {get; set;}
	}

	public List<FeedItem> GetAtomFeedItems(XDocument doc)
	{
		try
		{
			
			var feedItems = from item in doc.Root.Elements().Where(i => i.Name.LocalName == "entry")
						  select new FeedItem
						  {
							  Description = item.Elements().FirstOrDefault(i => i.Name.LocalName == "content").Value,
							  Link = item.Elements().FirstOrDefault(i => i.Name.LocalName == "link").Attribute("href").Value,
							  PublishDate = item.Elements().FirstOrDefault(i => i.Name.LocalName == "published").Value, //leaving parsing date here, to implement
							  Title = item.Elements().First(i => i.Name.LocalName == "title").Value
						  };
			return feedItems.ToList();
		}
		catch
		{
			return null;
		}
	}

	public List<FeedItem> GetRssFeedItems(XDocument doc)
	{
		try
		{
			
			var feedItems = from item in doc.Root.Descendants().First(i => i.Name.LocalName == "channel").Elements().Where(i => i.Name.LocalName == "item")
                           select new FEEDITEM
						  {
							  Description = item.Elements().First(i => i.Name.LocalName == "description").Value,
							  Link = item.Elements().First(i => i.Name.LocalName == "link").Value,
							  PublishDate = ParseDate(item.Elements().First(i => i.Name.LocalName == "published").Value),
							  Title = item.Elements().First(i => i.Name.LocalName == "title").Value
						  };
			return feedItems.ToList();
		}
		catch
		{
			return null;
		}
	}
   
	
 

Note

Above xml navigation is for rss elements. If feed is atom type the xml element names will vary. Code has to be modified with element names accordingly.
Here is the list, left values are for RSS2.0 and right values are for Atom 1.0
RSS 2.0 – Atom 1.0

rss –
channel – feed
title – title
link – link
description – subtitle
language –
copyright – rights
webMaster –
managingEditor – author or contributor
pubDate – published (in entry)
lastBuildDate (in channel) – updated
category – category
generator – generator
docs –
cloud –
ttl –
image – logo
– icon
rating –
textInput –
skipHours –
skipDays –
item – entry
author – author
– contributor
description – summary and/or content
comments –
enclosure –
guid – id
source –
– source

Ref: http://www.intertwingly.net/wiki/pie/Rss20AndAtom10Compared

Conclusion

Pretty easy and cool snippet. You will not get the error: ‘Element’ is an invalid XmlNodeType. If SyndicationFeed object is used, sometimes based on the xml invalid characters you will be getting error. Illegal characters might be ignored in browser but while parsing with SyndicationFeed we will have problem. Using XDocument will resolve everything. Just read each element of XDocment after loading

Useful links

Basic Linq To Xml queries – https://msdn.microsoft.com/en-us/library/bb943906.aspx
Ordering in LINQ – http://stackoverflow.com/questions/20436850/linq-to-xml-order-by-descending

October 5, 2015 · Adi · No Comments
Tags: , , , ,  · Posted in: C#, Sharepoint 2010, SharePoint 2013, Tools

Leave a Reply

What is 2 + 14 ?
Please leave these two fields as-is:
IMPORTANT! To be able to proceed, you need to solve the following simple math (so we know that you are a human) :-)