We have hit a situation where, when I run the REST call for http://165.113.16.5:8581/rest/devices I get the error 413. This says too much data returned. However, if I take that URL to my browser nothing happens - it just sits there. Several other REST calls work just fine - to eventrules, groups, consolidatedmonitoringprofiles. Is there a log or some other location, that will help me understand why my devices call no longer works? It worked fine yesterday. We only have 813 devices - that is not so many that it should fail to return results.
If volume is, truly, the problem is there a way to change the call so that I get "batches" where I can tell it to give me 1-200, 201-400, etc.?
I have had an Idea Submission created for this: Implement Throttle & Next into IM 2.0 RESTful Web Services
The DA log will show there is a ~200,000 rowcount limit on what can be returned. Why is something like '/rest/devices' returning > 200,000 rows? No idea. It started happening for me too with CAPM 2.5 and Support couldn't effectively answer that question.
However, this Idea Submission was created before the introduction of the OpenAPI. Now I feel the answer/solution is instead to begin using OpenAPI for these types of queries, and instead reserve the '/rest/' endpoints for more administrative actions (PUT, POST, etc.)
Thank you for your suggestion - I will give that a try.
so, I have been working on the alternative method and have run into a snag. I am able to get a URL to work - http://userid:password@165.113.16.5:8581/odata/api/devices in my chrome browser. However, when I do the same in my java program I get connection refused. Is there a different method for passing credentials? A config file, maybe?
Remove the "userid:password" from the URL, and instead look into how to pass a HTTP Header within your Java program. You need to pass the Header "Authorization".
https://en.wikipedia.org/wiki/Basic_access_authentication#Client_side
I have been working to make basic authentication work but am getting stuck with a 403 error that I cannot figure out. My code is below. If I remove the setrequestproperty I get a 401 so I am doing everything else properly. I have everything coded properly, I think, but something is definitely not right.
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
strURL = "http://165.113.16.5:8581/odata/api/devices?";
URL url = new URL(strURL);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
//the authentication is needed to use the OpenAPI request
String authHeader = strUserName + ":" + strPassword;
String encoded = new String(Base64.encodeBase64(authHeader.getBytes(), false));
connection.setRequestMethod("GET");
connection.setRequestProperty("Authorization", "Basic " + encoded);
int resp = connection.getResponseCode();
I found this exampe: http://stackoverflow.com/a/15555952
Maybe try changing encodeBase64() to just encode() ?
Everything else looks about the same though.
The following will give you manageable devices:
Do a GET from a REST interface to this URL:
http://<DA>:8581/rest/devices/manageable/
In the body use this "filter" XML:
<FilterSelect xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="filter.xsd">
<Select use="exclude" isa="exclude" relatesto="exclude">
<Item use="exclude">
<Name use="include"/></Item>
<Device use="exclude">
<PrimaryIPAddress use="include"/>
</Device>
</Select>
</FilterSelect>
I tried this and it worked fine for me in 2.6.
HTH,
Joe
Well, progress is being made. I used URL http://165.113.16.5:8581/rest/devices/manageable and got through the authentication issues. I had an error in my authentication logic that I correct. Now, however, I am getting a 400 error returned. The exact error returned is below:
java.io.IOException: Server returned HTTP response code: 400 for URL: http://165.113.16.5:8581/odata/api/devices
When I make the request in chrome I get what is shown below (which appears to be correct) - when I try to view in IE or Firefox the browsers think this is a RSS feed (which it is not, of course). Could this be what is happening in my program and, if so, is there a way to avoid it? I only pasted the top portion of that being returned because there is way too much data to paste it all.
************************ RESULTS OF URL REQUEST WHEN RUN IN CHROME *****************************************
<feed xmlns="http://www.w3.org/2005/Atom" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xml:base="http://165.113.15.211:8581/odata/api/">
<title type="text">devices</title>
<updated>2015-12-14T11:38:45.919-05:00</updated>
</author>
<link href="devices" rel="self" title="devices"/>
<m:count>11887</m:count>
<category term="com.ca.im.odata.device" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme"/>
<link href="devices(5L)" rel="edit" title="device"/>
<link href="devices(5L)/metricfamilyhistories" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/metricfamilyhistories" title="metricfamilyhistories" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/components" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/components" title="components" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/cpus" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/cpus" title="cpus" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/memories" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/memories" title="memories" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/interfaces" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/interfaces" title="interfaces" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/discoveryprofile" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/discoveryprofile" title="discoveryprofile" type="application/atom+xml;type=entry"/>
<link href="devices(5L)/groups" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/groups" title="groups" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/qosclassmaps" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/qosclassmaps" title="qosclassmaps" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/qospolicers" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/qospolicers" title="qospolicers" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/qoscontracts" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/qoscontracts" title="qoscontracts" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/qosqueuings" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/qosqueuings" title="qosqueuings" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/qosreds" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/qosreds" title="qosreds" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/qostrafficshapings" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/qostrafficshapings" title="qostrafficshapings" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/switchingenginemfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/switchingenginemfs" title="switchingenginemfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/dataloadermfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/dataloadermfs" title="dataloadermfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/mplssegmentinmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/mplssegmentinmfs" title="mplssegmentinmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/fibrechannelif2mfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/fibrechannelif2mfs" title="fibrechannelif2mfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/virtualcentersmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/virtualcentersmfs" title="virtualcentersmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/qoscospolicermfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/qoscospolicermfs" title="qoscospolicermfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/qoscosqueuemfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/qoscosqueuemfs" title="qoscosqueuemfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/devicepollingstatisticsmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/devicepollingstatisticsmfs" title="devicepollingstatisticsmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/vmwarehostdisksmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/vmwarehostdisksmfs" title="vmwarehostdisksmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/mplsinterfacemfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/mplsinterfacemfs" title="mplsinterfacemfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/vmwareclustermfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/vmwareclustermfs" title="vmwareclustermfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/mplssegmentoutmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/mplssegmentoutmfs" title="mplssegmentoutmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/reachabilitymfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/reachabilitymfs" title="reachabilitymfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/maintenancetasksmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/maintenancetasksmfs" title="maintenancetasksmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/genericdiskmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/genericdiskmfs" title="genericdiskmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/hostaggregatemetricsmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/hostaggregatemetricsmfs" title="hostaggregatemetricsmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/vmwaredatacentermfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/vmwaredatacentermfs" title="vmwaredatacentermfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/partitionsmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/partitionsmfs" title="partitionsmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/datacollectorsystemstatisticsmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/datacollectorsystemstatisticsmfs" title="datacollectorsystemstatisticsmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/eventmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/eventmfs" title="eventmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/vmwarevirtualdisksmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/vmwarevirtualdisksmfs" title="vmwarevirtualdisksmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/nav_systemstatisticsmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/nav_systemstatisticsmfs" title="nav_systemstatisticsmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/datacollectorpollingstatisticsmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/datacollectorpollingstatisticsmfs" title="datacollectorpollingstatisticsmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/daactivemqbrokerhealthmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/daactivemqbrokerhealthmfs" title="daactivemqbrokerhealthmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/rollupmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/rollupmfs" title="rollupmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/fibrechannelif1mfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/fibrechannelif1mfs" title="fibrechannelif1mfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/virtualmemorymfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/virtualmemorymfs" title="virtualmemorymfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/qoscoscontractmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/qoscoscontractmfs" title="qoscoscontractmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/cpumfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/cpumfs" title="cpumfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/vmwareresourcepoolsmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/vmwareresourcepoolsmfs" title="vmwareresourcepoolsmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/baselinemfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/baselinemfs" title="baselinemfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/icmpmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/icmpmfs" title="icmpmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/memorymfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/memorymfs" title="memorymfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/portmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/portmfs" title="portmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/availabilitymfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/availabilitymfs" title="availabilitymfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/ribquerymfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/ribquerymfs" title="ribquerymfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/dajvmhealthmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/dajvmhealthmfs" title="dajvmhealthmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/fwconnstatsmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/fwconnstatsmfs" title="fwconnstatsmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/systemstatisticsmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/systemstatisticsmfs" title="systemstatisticsmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/virtualmachineaggregatemetricsmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/virtualmachineaggregatemetricsmfs" title="virtualmachineaggregatemetricsmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/qosclassmapmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/qosclassmapmfs" title="qosclassmapmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/nav_netscalermfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/nav_netscalermfs" title="nav_netscalermfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/openapiquerymfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/openapiquerymfs" title="openapiquerymfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/dhcpmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/dhcpmfs" title="dhcpmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/icmpjittermfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/icmpjittermfs" title="icmpjittermfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/pathechomfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/pathechomfs" title="pathechomfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/tcpmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/tcpmfs" title="tcpmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/dlswmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/dlswmfs" title="dlswmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/ethernetpingmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/ethernetpingmfs" title="ethernetpingmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/udpmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/udpmfs" title="udpmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/ethernetjittermfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/ethernetjittermfs" title="ethernetjittermfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/rttmonhttpmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/rttmonhttpmfs" title="rttmonhttpmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/dnsmfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/dnsmfs" title="dnsmfs" type="application/atom+xml;type=feed"/>
<link href="devices(5L)/jittermfs" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/jittermfs" title="jittermfs" type="application/atom+xml;type=feed"/>
<d:IsAlso m:null="true"/>
<d:DisplayName>DataAggregator:165.113.15.211</d:DisplayName>
<d:Name>localhost.localdomain</d:Name>
<d:DisplayDescription m:null="true"/>
<d:CreateTime>2015-10-30T07:42:46.904-04:00</d:CreateTime>
<d:MDRItemID m:null="true"/>
<d:IPDomainID>2</d:IPDomainID>
<d:ConsolidatedMonitoringProfile>167</d:ConsolidatedMonitoringProfile>
<d:PrimaryIPAddress>127.0.0.1</d:PrimaryIPAddress>
<d:SupportsOnDemandMFDiscovery>true</d:SupportsOnDemandMFDiscovery>
<d:SupportedProtocols>SelfMonitor</d:SupportedProtocols>
<d:DiscProfileID m:null="true"/>
<d:HostName>localhost.localdomain</d:HostName>
<d:IPInformation m:null="true"/>
<d:ItemSubType m:null="true"/>
<d:ItemType m:null="true"/>
<d:Latitude m:null="true"/>
<d:Longitude m:null="true"/>
<d:Elevation m:null="true"/>
<d:LocationDesc m:null="true"/>
</m:properties>
</content>
</entry>
Hello Edward,
Its been a while since we've seen further updates for this forum thread from anyone.
I wonder does it remain an issue for you or were you able to find a solution to this problem? If it remains an issue let us know so we can try to finish figuring out a solution for you.
Thanks,
Michael