SharePoint Blogs / SharePoint University
SharePoint Blogs and SharePoint University - all in one place!
Need SharePoint Training? Attend a SharePoint Bootcamp!

Please delete cookies related to sharepointblogs.com and sharepointu.com to resolve login issues!

Accessing SharePoint List Data as XML

The other day, I found myself in need of a way to access SharePoint list data as XML but the only method available to me was a simple http GET.  This is not too bad, but what can I get from SharePoint via GET?  RSS is too limited and scraping the page is too painful and error-prone to even contemplate. 

It turns out that there is an option available in 2007 that was carried forward from SharePoint 2003 / FrontPage days: owssvr.dll.  But this isn't some forgotten FrontPage artifact, it is still a central part of SharePoint.  In fact, if you pull up a list view and then view the source looking for owssvr.dll, you will see that this is the mechanism behind both the Export to Spreadsheet and Open with Access options on the list Actions menu.

How It Works

Simply put together a URL like this:

http://MyServer/[site]/_vti_bin/owssvr.dll?Cmd=Display&List={listGuid}&XMLDATA=TRUE

This will only return the fields that are defined on the default view of the list.  If you need specific fields then you need to create a view with those fields and pass the View ID as well, like this:

http://.../owssvr.dll?Cmd=Display&List={listGuid}&view={viewGuid}&XMLDATA=TRUE

Specifying Fields to be Returned

There is also a Query parameter that lets you specify which fields are to be included in the resulting XML, regardless of how the view is defined.  For example, if you wanted just to bring back the Title and Status Fields, you would add the field names separated by spaces (URL Encoded, of course) like "&Query=Title%20Status".  If you want to return all fields, use an asterisk (*) instead of field names.

http://.../owssvr.dll?Cmd=Display&List={listGuid}&query=Title%20Status&XMLDATA=TRUE

Filtering Data

Regardless of whether you pass a view or use the default it will still use the filter defined by that view.  Not bad, but you can trim this data even more by including a filter of your own using the FilterFieldn and FilterValuen arguments in the querystring.  These are the same values that are passed when you use the filter options in the column headers of a view which makes it pretty easy to track down exactly what needs to be passed.  Simply pull up the view that is your starting point and use the column filters to create your desired filter.  Once you have it, grab all of the FilterField and FilterValue items from the querystring and add them on to yours.

http://.../owssvr.dll?Cmd=Display&List={listGuid}&query=Title%20Status&XMLDATA=TRUE&FilterField1=Status&FilterValue1=In%20Progress

For a full list of what can be done with this technique, check out the URL Protocol or the older Using the URL Protocol on MSDN.


Posted 01-10-2008 7:02 AM by DavidWise

Comments

Links (1/10/2007) « Steve Pietrek’s SharePoint Stuff wrote Links (1/10/2007) « Steve Pietrek’s SharePoint Stuff
on 01-10-2008 7:19 PM

Pingback from  Links (1/10/2007) « Steve Pietrek’s SharePoint Stuff

double density design - » Bookmarking the web - w02/2008 wrote double density design - » Bookmarking the web - w02/2008
on 01-11-2008 1:24 PM

Pingback from  double density design  -   » Bookmarking the web - w02/2008

judy osnoe wrote re: Accessing SharePoint List Data as XML
on 01-11-2008 2:20 PM

can we get this to work with blogs?  the out of the box reader does not support authenticated feeds and problem is anon access is not something my client feels comfortable with.   I've tried sp360 reader webpart but it won't accept the blog feed url as even being valid.   so I came across this which seems to be an alternative over rss anyway.  

I look forward to hearing from you

Judy

DavidWise wrote re: Accessing SharePoint List Data as XML
on 01-11-2008 3:04 PM

Judy,

I'd have to say no as even this requires normal Sharepoint Authentication.

What you could probably do is create an "rss.aspx" page somewhere that accepts an anonymous request, then creates a new WebRequest using a NetworkCredential that is recognized by SharePoint, fetches the SharePoint RSS feed, then streams that directly back to the caller.  Not pretty perhaps, but simple and quick to implement.

Good Luck!

Sarah T-O wrote re: Accessing SharePoint List Data as XML
on 01-14-2008 10:36 AM

I can't get this to work on a List that contains a Lookup field.  I just get a blank page in the browser. Have you encountered this too? Any ideas?

DavidWise wrote re: Accessing SharePoint List Data as XML
on 01-14-2008 6:13 PM

Sarah,

Try doing a View Source on that blank page.  I bet it is chock full-o-xml :)

Ken Pespisa wrote re: Accessing SharePoint List Data as XML
on 02-06-2008 8:21 AM

Thank you!  This is a great tip for developers.

Pedro Mateus wrote re: Accessing SharePoint List Data as XML
on 02-25-2008 11:36 AM

can i filter the data that goes exported to EXCEL? For XML works but not for EXCEL...

Other aproach, importing xml in EXCEL using this builded url and after i add item in list, in excel i do refresh i get error "An xml table cannot be resized to accomodate the data"

Brad wrote re: Accessing SharePoint List Data as XML
on 04-10-2008 12:51 PM

Where do you get the listGUID?

DavidWise wrote re: Accessing SharePoint List Data as XML
on 04-10-2008 4:36 PM

Pedro,  I'm not sure how this would work with Excel as I haven't had the need to look into it.  I do imagine that the experience might vary greatly between Excel 2003 and 2007.  Good luck,

DavidWise wrote re: Accessing SharePoint List Data as XML
on 04-10-2008 4:49 PM

Brad,  The list GUID is very easy to get.  Simply open a list view and do a View Source.  Look for "List=" in the source.  In most sites, the first one you find will contain the UrlEncoded GUID pretty high up in the page.  UrlDecode that and you are all set.

You can also pull the list GUID from the URL of  a system page whenever you take action that directs you to one, like by clicking "Modify This View" from the views menu of the list.

Hope that helps.

Christophe wrote re: Accessing SharePoint List Data as XML
on 05-14-2008 3:46 AM

David, I think you lost your bet. I am experiencing the same issue as Sarah on lists with lookup fields, and "view source" just shows an empty page...

I actually came across your post because I was doing a search on this issue.

Brett wrote re: Accessing SharePoint List Data as XML
on 05-23-2008 9:40 AM

I have used this technique but I am having difficulty with requesting field names that have spaces in them. For example I need a field to come back in the XML called "Work Request Name".  If fields are separated by spaces does anyone know the syntax to specifying this name on the URL?  Assume, I do not have rights to change the name or create views.  One would think it would be something like "Query=Name%20Status%20Work_Request_Name&XMLDATA=True" ...  unfortunately that doesnt work.  Anyone a method?

DavidWise wrote re: Accessing SharePoint List Data as XML
on 05-28-2008 5:01 PM

Brett,  Fields have several names.  The one you normally see has a space in it, but you probably need to refer to the field using its internal name.

By default, SharePoint will create a field name based on the display name you entered when you created the field.  However, it will replace all spaces with "_x0020_" and will limit the name to 32 characters.  Thus, if you have a long name or one with lots of spaces, it will get butchered even more than you might suspect.

The easiest way to find the real name is to sort a view by the column you are looking for.  When you do this, it adds the internal field name to the querystring in the SortField parameter.

You will need to UrlEncode the internal field name for use in the querystring which makes a space-laden field name just about as nasty as it can get short of Base64 encoding it ;)

Brett wrote re: Accessing SharePoint List Data as XML
on 05-29-2008 3:09 PM

David, awesome help! Thanks so much! After doing what you recommended I was able to further expand on a technique.  So what I have found worked well is to simply run:

http://[yourserver]/_vti_bin/owssvr.dll?Cmd=Display&List={your list GUID}&Query=*&XMLDATA=TRUE

That will then return you all of the avaialbe fields in the list providing the name="..."  for each one of them.

So for example up in the schema portion of the response you will have the list of all the field attributes, for example:

<s:AttributeType name="ows_Additional_x0020_Detail_x0020_Ne" rs:name="Additional Detail Needed?" rs:number="9">

Now it SEEMS you can simply eliminate the preceding "ows_" and then replace all of the _ with %5f hex values.  Therefore the field in this case would be referenced on the URL as:

Additional%5fx0020%5fDetail%5fx0020%5fNe

in the URL.  Now I also noticed when making the SOAP webService call using the section ViewFields you simply remove the preceding ows_ and did not UrlEncode the _:

<ViewFields>

<FieldRef name="Additional_x0020_Detail_x0020_Ne" />

</ViewFields>

Again thanks for the great help and hopefully others find this useful!

DavidWise wrote re: Accessing SharePoint List Data as XML
on 05-30-2008 10:45 AM

Brett, the "*" option is great for lists with only a few columns but in lists with 75+ columns (like most of the ones I seem to deal with) that brings back way more data than I need.

I'm glad it is working for you, though!

Brett wrote re: Accessing SharePoint List Data as XML
on 06-02-2008 12:58 PM

Right, the lists I am dealing with have more than 75+ as well.  That is really why I needed the specific field listing technique.  I was only using the "*" option to quickly get all the internal names of the columns available for potentially listing.

Thanks again!

Joe wrote re: Accessing SharePoint List Data as XML
on 06-12-2008 12:57 PM

Is there a way to figure out the link to an attached file using this approach? Because when this generates the XML it shows that there is an attachment but does not show the URL to the attachment.

Thanks! Hope you can help me with this dilemma

Ani wrote re: Accessing SharePoint List Data as XML
on 07-06-2008 3:29 PM

Hello David

I have use various format as mentioned by various authos above. I tested each different options and it is working correctly, only problem I notieced that every xml is returning only first 18 columns, and in most of the views I am having 20-22 column, but it is returning only first 18 columns.

Anything I am missing here?

Avi

Ani wrote re: Accessing SharePoint List Data as XML
on 07-06-2008 3:36 PM

One more thing I noticed, that I tested with diferent views and everytimes it is returning same columns in xml i.e.  only 18 columns, infact in 1 view there are only 4 column, but when I displat its xml it is returning all 18 columns.

ANy clue

Scott wrote re: Accessing SharePoint List Data as XML
on 07-14-2008 3:30 PM

Sarah T-O:

I found that it is not an issue of being a lookup collum but if you allow multiple selections.

Any one know any work arounds for this?

Tony wrote re: Accessing SharePoint List Data as XML
on 08-12-2008 4:57 PM

How do you retrieve items in a folder using this method please?

Using the RPC method via the URL Protocol « Path to SharePoint wrote Using the RPC method via the URL Protocol &laquo; Path to SharePoint
on 09-23-2008 9:45 PM

Pingback from  Using the RPC method via the URL Protocol &laquo; Path to SharePoint

judster7129 wrote re: Accessing SharePoint List Data as XML
on 11-06-2008 3:24 AM

Hi David

I've browsing thru the comments on your blog and found them very interesting. I also need some help when it comes to sharepoint.

judster7129 wrote re: Accessing SharePoint List Data as XML
on 11-06-2008 3:32 AM

i am retrieving data from a list in sharepoint as OuterXml. i specified the fields i want and created a dataset and a datatable which i bind to the dataset.

i then loop thru the data in the list and display the fields which i bind to a repeater. what i find though is that one of the fields doesnt keep the format of the data as captured/stored in sharepoint. is there a way i can format the xml to look exactly as on sharepoint itself.  here is a snippet of my code:

listService = new GITS.Lists();

               listService.Credentials = new System.Net.NetworkCredential("username", "password", "domain");

               listService.Url = "mysite/.../Lists.asmx";

               xmlDoc = new System.Xml.XmlDocument();

               viewFields = xmlDoc.CreateElement("ViewFields");

               query = xmlDoc.CreateNode(XmlNodeType.Element, "Query", "");

               queryOptions = xmlDoc.CreateNode(XmlNodeType.Element, "QueryOptions", "");

               query.InnerXml = "<Where><Eq><FieldRef Name='Meeting_x0020_Date' /><Value Type= 'DateTime'>" + sDate + "</Value></Eq></Where>";

               queryOptions.InnerXml = "<IncludeAttachmentUrls>TRUE</IncludeAttachmentUrls>";

               viewFields.InnerXml = "<FieldRef Name='Report_x0020_Subject'/><FieldRef Name='Agenda_x0020_Item_x0020_No_x002e'/><FieldRef Name='Attachments'/><FieldRef Name='Resolution_x0020_Comments'/>";

               nodeListItems = listService.GetListItems(strSubCouncilName, null, query, viewFields, "5000", queryOptions, null);

               DataSet _ds     = new DataSet();

               DataTable dt    = new DataTable();

               string _xml = nodeListItems.OuterXml;

               xmlDocResult        = new XmlDataDocument();

               XmlDocument _doc    = new XmlDocument();

               XmlDataDocument xdd = new XmlDataDocument();

               xdd.LoadXml(nodeListItems.OuterXml);

               _ds.Tables.Add("newTableName");

               dt = _ds.Tables["newTableName"];          

               dt.Columns.Add("ResolutionDetails", typeof(String));

               XmlNodeList rows = xdd.GetElementsByTagName("z:row");

               foreach (XmlNode listItem in rows)

               {

                   DataRow newRow = dt.NewRow();

                   if (listItem.Attributes["ows_Resolution_x0020_Comments"] != null)

                   {

                       newRow["ResolutionDetails"] = listItem.Attributes["ows_Resolution_x0020_Comments"].Value;

                   }

                   dt.Rows.Add(newRow);

               }

               ResultsRepeater.DataSource = dt;

               ResultsRepeater.DataBind();

           }

judster7129 wrote re: Accessing SharePoint List Data as XML
on 11-06-2008 5:31 AM

An example of how the data displays on the web:

Below how it was captured/displayed orginally in sharepoint:

This is the FIRST paragraph. This is the FIRST paragraph. This is the FIRST paragraph. This is the FIRST paragraph. This is the FIRST paragraph. This is the FIRST paragraph. This is the FIRST paragraph. This is the FIRST paragraph.

This is the SECOND paragraph. This is the SECOND paragraph. This is the SECOND paragraph. This is the SECOND paragraph. This is the SECOND paragraph.

When it displays on the web it displays like this:

This is the FIRST paragraph. This is the FIRST paragraph. This is the FIRST paragraph. This is the FIRST paragraph. This is the FIRST paragraph. This is the FIRST paragraph. This is the FIRST paragraph. This is the FIRST paragraph. This is the SECOND paragraph. This is the SECOND paragraph. This is the SECOND paragraph. This is the SECOND paragraph. This is the SECOND paragraph.

DavidWise wrote re: Accessing SharePoint List Data as XML
on 11-06-2008 3:19 PM

Judster,

My guess would be that you are not formatting the results in a way that is accounting for any hidden LineFeed characters that might be in the original data.  Html treats linefeeds as simple whitespace.  You will probably need to replace them with <br /> tags.

vinay wrote re: Accessing SharePoint List Data as XML
on 11-21-2008 2:24 AM

Using the above approach data of only first page (100 items) are returned, How can I get all the records in one go?

Steph wrote re: Accessing SharePoint List Data as XML
on 12-04-2008 11:36 AM

I'm trying to use this in an xml webpart so i can apply an xsl to a list. But when i test the link it works fine but after applying the page says it can't load the xml.

ShaunM wrote re: Accessing SharePoint List Data as XML
on 12-07-2008 4:03 PM

I am wondering how this can be extended to include the structure of subfolders within a list. for example:

<ds:data>

  <z:row 'some file'

  <z:row 'some file'

  <z:row 'some folder'

       <z:row 'some file'

       <z:row 'some file'

  <z:row 'some folder'

       <z:row 'some file'

       <z:row 'some file'

Peter Levine wrote re: Accessing SharePoint List Data as XML
on 01-05-2009 1:15 PM

Hi,

I'm getting the following error message when I try to access a SharePoint list via the URL method. But the page exists. The GUID I'm using is the one I see on that page.

Here's the error:

The page you selected contains a list that does not exist.  It may have been deleted by another user.  Click "Home" at the top of the page to return to your Web site.

Pete

Toast wrote re: Accessing SharePoint List Data as XML
on 01-08-2009 8:21 AM

I'm looking to use this approach and we've a field in our list which is set to "append to existing data" we've configured the view so that the field is part of the view, though it only shows the last entry in the field and not the entire contents? i.e. if I have an item and enter "some information" in to field, and then edit this item, and change a different field when I use this method (or indeed the export to excel method) nothing is returned where we'd expect "some information" to be shown?

has anyone come accross this before, or have any tips on how to get all of the contents of this field to be included in the export?

John wrote re: Accessing SharePoint List Data as XML
on 02-13-2009 2:30 PM

Great post, just what I was looking for. Note: the parameters are case sensitive. This may help others like avoid some frustration.

Jamie wrote re: Accessing SharePoint List Data as XML
on 03-10-2009 3:26 PM

How are folders and sub-folders dealt with using this?

Rich wrote re: Accessing SharePoint List Data as XML
on 03-11-2009 2:18 AM

I'm operating in an evironment where I have several different site collections.  I need to be able to pull a filtered version of an All Items view from a list in one site collection to a site in another site collection.  The only way I can figure to pull that information is by building custom views for each of the list views I need to pull over, and then moving the data via an RSS view.  Any suggestions on another cleaner, more flexible way of doing this?

Chris V wrote re: Accessing SharePoint List Data as XML
on 03-17-2009 4:32 PM

Why am I having issues viewing the data in the "Data Source Details" in Designer.  I have no issues viewing it in my browser.

Gorkem Tolan wrote re: Accessing SharePoint List Data as XML
on 04-15-2009 10:20 AM

I am a beginner in Sharepoint. I am wondering whether I can write data into sharepoint list from an XML file or external website. I would appreciate for your comments and suggestions.

Zack wrote re: Accessing SharePoint List Data as XML
on 06-04-2009 12:55 AM

Does anyone know what features use XML as their transport layer in SharePoint and/or What APIs are provided by SharePoint for 3rd parties?

Micheal wrote re: Accessing SharePoint List Data as XML
on 06-17-2009 12:44 PM

I'm having the same issue as Sarah, Christophe, and Scott.  I get nothing but a blank page for lists that have multiselect lookups.  View Source shows nothing.  Has anyone found a way to get this to work?

Steve wrote re: Accessing SharePoint List Data as XML
on 06-19-2009 1:19 PM

I've added the RootFolder parameter to the url to get a subfolder so for example to get a folder or subfolder you can use the link

http://<portal>/<sites>/_vti_bin/owssvr.dll?Cmd=Display&List={GUID}&RootFolder=<url path to Folder or SubFolder>&XMLDATA=TRUE

also, if you want to query specific fields just add the Query=*, Query=<Field> parameter to the URL.

Add a Comment

(required)  
(optional)
(required)  
Remember Me?
Need SharePoint Training? Attend a SharePoint Bootcamp!
Posts (c) their respective authors. Everything else (c) 2009 SharePoint Experts, Inc.