Rss Feed throws "WebException" please help!

Discussion in 'Visual Studio' started by albanello, Jun 18, 2019.

  1. ********
    The code shown below "Code executed" is used for 3 other rss feeds and it works without any problem!
    BUT
    "Exception thrown" for the "URL string passed" (Both shown below)

    I have tried to isolate the "Code executed" where the problem occurs to make it easier for someone to help me!
    I have also tried to isolate the "Rss feed data"! (Both shown below)

    I have been tring to figure this out for several months with no success. (Maunally entering Rss feed data to my Web application)
    PLEASE help!

    ********
    URL string passed
    *
    strURL = "https://www.superpages.com/lottery/rss/calottery.xml"
    THIS strURL "Rss feed data" is the ONLY Rss feed that has

    ********
    Code executed
    *
    Try
    Dim rssReq As WebRequest = WebRequest.Create(strURL)

    Dim rssProxy As New WebProxy(strURL, True)

    rssReq.Proxy = rssProxy
    rssReq.Timeout = 5000

    'This is where Exception is thrown
    'at this line of code
    >> Dim rep As WebResponse = rssReq.GetResponse()

    Dim xtr As New XmlTextReader(rep.GetResponseStream())

    dstRssFeed.ReadXml(xtr)

    xtr.Close()

    rep.Close()

    Catch ex0 As Exception
    "Exception thrown" (See below)

    End Try

    ********
    Exception thrown
    *
    System.Net.WebException was caught
    Message="The remote name could not be resolved: 'www.superpages.comwww.superpages.comhttp'"
    Source="System"
    StackTrace:
    at System.Net.HttpWebRequest.GetResponse()
    at LotteryApp.RssFeedClass..ctor(String strURL_arg, Int32 intRow_arg, String strLottery_arg) in C:\Projects\ASP_NET\Lottery\Application\MyWebAppProj\LotteryApp\Maintenance\RssFeedClass.vb:line 59
    InnerException:

    ********
    Rss feed data
    *
    <?xml version="1.0" encoding="utf-8"?>
    <rss version="2.0">
    <channel>
    <title>California Lottery Results from SuperPages.com</title>
    <description>California Lottery Results from SuperPages.com: White Pages, and Online Directories</description>
    <link>https://www.superpages.com/cities/lottery/lottery-CA.html</link>
    <lastBuildDate>Fri, 07 Jun 2019 12:27:13 GMT</lastBuildDate>
    <docs>http://blogs.law.harvard.edu/tech/rss</docs>
    <generator>Feed for Node.js</generator>
    .
    .
    .
    <item>
    <title><![CDATA[SuperLotto Plus winning results for 06/05/2019]]></title>
    <link>https://www.superpages.com/cities/lottery/lottery-CA.html</link>
    <guid>https://www.superpages.com/cities/lottery/lottery-CA.html</guid>
    <description><![CDATA[SuperLotto Plus winning results for 06/05/2019: 12-16-21-28-41, Mega Ball: 26]]></description>
    </item>
    .
    .
    .
    </channel>
    </rss>
     
  2. I have NOT received any reply. Have I posted to the incorrect forum? Should I post to ASP.NET forum? can anybody suggest a different forum?
    Thanks
     
  3. martino

    martino DiscountASP.NET Staff

    Your URL Is incorrect. Do you have something in your code that is making the URL repeat itself? Do you have the incorrect URL in your call to that page?
     
  4. ********
    Frank:
    Thank you for your response.
    Please refer to my initial post.

    ********
    Martino:
    "Your URL Is incorrect."

    Frank:
    - Refer to my initial post> "URL string passed"
    strURL = "https://www.superpages.com/lottery/rss/calottery.xml"
    I believe this to be the correct URL. It can be verified by:
    1) Enter URL in a Browser URL window
    THEN
    2) Select "View Source" and you will see the full xml Rss Feed

    My original post had a trimmed down version to simplify for the example.

    ********
    Martino:
    "Do you have something in your code that is making the URL repeat itself?"

    Frank:
    - Refer to my initial post> "Code executed"
    Not to my knowledge. I am not aware of this code executing multiple times! I have stepped through with the debugger.

    ********
    Martino:
    "Do you have the incorrect URL in your call to that page?"

    Frank:
    - Refer to my initial post> "Code executed"
    - No, I have no idea why "Dim rep As WebResponse = rssReq.GetResponse()" is throwing the
    System.Net.WebException
    Message="The remote name could not be resolved: 'www.superpages.comwww.superpages.comhttp'"
    OR
    - How it is coming up with the URL string it has!

    ********
    Frank:
    Thanks again for your help. I have been trying to figure this out for Months.
     
  5. RayH

    RayH DiscountASP.NET Lackey DiscountASP.NET Staff

    There's something in your code/configuration that is causing the URL to be repeated. Unfortunately, I don't see anything in the code you provided that could cause the problem. The only thing I noticed in your RSS feed is that you have the <link> node declared twice. The code that is causing the problem might not be listed, or it might be something in your web.config file that is causing the issue.
     
    martino likes this.
  6. martino

    martino DiscountASP.NET Staff

    Yes, exactly what RayH said.

    The problem is the host URL is being repeated in the string.
     
  7. *******************************************************************************
    *******************************************************************************
    RayH:
    "The only thing I noticed in your RSS feed is that you have the <link> node declared twice."

    Frank:
    - "Their" Rss Feed is constructed by "Them" I do not add anything to it.
    - I only see the <link> node once in each <item> node

    RayH and Martino:
    "There's something in your code/configuration that is causing the URL to be repeated. Unfortunately, I don't see anything in the code you provided that could cause the problem. The code that is causing the problem might not be listed, or it might be something in your web.config file that is causing the issue."

    Frank:
    - Refer to my initial post> "Code executed"
    ! This exact same code is used to acquire the same data from other Lottery Rss Feeds.
    - Refer to my initial post> "Rss feed data"
    ! The only different I see in the Rss Feed, that has the problem, is that the problem Rss Feed has some ![CDATA[........]] in its <title> and <description> Rss Feed <item> nodes

    ********
    Frank:
    Thanks again for your help.

    *******************************************************************************
    *******************************************************************************
     
  8. RayH

    RayH DiscountASP.NET Lackey DiscountASP.NET Staff

    Do you have a link to another RSS feed that is working that we can compare it to?
     
    martino likes this.
  9. RayH

    Here is a link that uses the exact same code, I show in my original post, and it works with no problem.
    "http://www.molottery.com/rss-num.xml"

    Note this Rss Feed does not have any ![CDATA[........]].

    Thanks for your help
     
  10. RayH

    RayH DiscountASP.NET Lackey DiscountASP.NET Staff

    Well, what you can try is copy the XML file, place it on your site and remove the ![CDATA[........]] lines. Then have your code call the XML file on your site. If it doesn't error out, then you know your code is unable to process those lines. You might also want to try remove the <guid> lines as well.
     
    martino likes this.
  11. albanello:
    RayH and Martino sorry it took me so long to get back to you, my brain is fried trying to figure out how to get this code to work!

    ********
    RayH:
    "Well, what you can try is copy the XML file, place it on your site and remove the ![CDATA[........]] lines"

    *
    albanello:
    I got the Rss file by: (Locally)
    - Entering in Internet Explorer the URL "http://www.superpages.com/lottery/rss/calottery.xml"
    - Select "View->Source"
    - Copy/Paste Source to my Web Site in the "App_Data" directory! (file name "CSLPRssFeed.xml")

    ********
    RayH:
    "Then have your code call the XML file on your site."

    *
    albanello:
    - On my site, used "CSLPRssFeed.xml", below code worked without having to removing "CDATA" stuff
    Note:
    - All actual directories in path are not displayed

    dim strURL as string = "C:\Dir_0\Dir_1\Dir_2\Dir_3\Dir_4\Dir_5\App_Data\CSLPRssFeed.xml"

    xtr = New XmlTextReader(strURL)
    dstRssFeed.ReadXml(xtr)

    - Exact same code DOES NOT work with
    strURL = "http://www.superpages.com/lottery/rss/calottery.xml"

    I think it does not work because the ![CDATA[........]] stuff is not sent to the "XmlTextReader()".
    I CAN'T think of a way to remove the ![CDATA[........]] stuff when using the URL "http://www.superpages.com/lottery/rss/calottery.xml"


    Any other suggestion will be appreciated.
    Thanks for your help

    albanello
     
  12. RayH

    RayH DiscountASP.NET Lackey DiscountASP.NET Staff

    Please read this regarding CDATA. And then compare the two links you gave us:

    http://www.molottery.com/rss-num.xml
    https://www.superpages.com/lottery/rss/calottery.xml

    Note, the first link is returning valid XML. The second link is returning literal characters. That's probably why it's not being processed correctly. Microsoft also says XmlTextReader does not perform any data validation.

    So, it works when you reference the file using the file system because it's reading it as XML. It doesn't work when use you HTTP because what it's seeing is the result of GET request (i.e. it's trying to read what you see on the page in the second link). That's probably why it's returning the URL multiple times.
     
    Last edited: Jul 18, 2019
    martino likes this.
  13. albanello:
    As with all the other "reader" attempts I still get the same result:
    - CSLP URL "http://www.superpages.com/lottery/rss/calottery.xml" Fails every time (Throws exception see below)
    - CSLP URL by manually creating the Xml file: (Locally) Works every time as expected
    ! Entering in Internet Explorer the URL "http://www.superpages.com/lottery/rss/calottery.xml"
    ! Select "View->Source"
    ! Copy/Paste "Source" to my Web Site in the "App_Data" directory! (file name "CSLPRssFeed.xml")
    - "http://www.molottery.com/rss-num.xml" Works every time as expected

    ********
    RayH:
    "Please read this regarding CDATA."

    albanello:
    I read everything you suggested I think I under stand (CDATA) it is a way of commenting out text in a xml file. How to get the data to appear in a rss feed file when it is downloaded I have no idea. (Sorry my experience/skill level is lacking)

    ********
    RayH:
    "And then compare the two links you gave us:

    http://www.molottery.com/rss-num.xml
    https://www.superpages.com/lottery/rss/calottery.xml

    Note, the first link is returning valid XML. The second link is returning literal characters. That's probably why it's not being processed correctly."

    albanello:
    You refer to the second link returning "literal" I would agree but I think it is NOT returning the commented out sections which is what is needed. I don't know! ?
    I am ready to give up and try and go for a work around solution. I think "superpages" created a Rss feed to not work like all other Rss Feeds intentionally to limit it to just one use, display data only using their software. They say they are providing a free rss feed.

    ********
    RayH:
    "Microsoft also says XmlTextReader does not perform any data validation."

    albanello:
    I reworked the code to use the "XmlReader". Below code is in its simplest form because I have NO idea how to configure the "settings" with all the possibilities available. What I don't understand is why the locally created/stored xml file works........it is a exact duplicate of the original xml file. I am using (I guess) all the default "settings" because my skill level is no where near the level to determine what configurations settings to use and what they should be set at.

    The below code:
    - Throws Exception when "strURL" is "https://www.superpages.com/lottery/rss/calottery.xml"
    - Does Not Throws Exception when "strURL" is "http://www.molottery.com/rss-num.xml"
    - Does Not Throws Exception when "strURL" uses the manually created and locally stored xml file Note: "CDATA" markers are still in this xml file! Still works?

    Code that was run to get the above listed resulting (WebException shown below):
    '!!!!!!!!
    'Create the XmlReader object.
    Dim myXmlReader As XmlReader = XmlReader.Create(strURL)

    'Fill DataSet with XML data
    dstRssFeed.ReadXml(myXmlReader)
    '!!!!!!!!

    Here is the WebException thrown: (Message="An existing connection was forcibly closed by the remote host")
    *
    System.Net.WebException was caught
    Message="The underlying connection was closed: An unexpected error occurred on a send."
    Source="System"
    StackTrace:
    at System.Net.HttpWebRequest.GetResponse()
    at System.Xml.XmlDownloadManager.GetNonFileStream(Uri uri, ICredentials credentials)
    at System.Xml.XmlDownloadManager.GetStream(Uri uri, ICredentials credentials)
    at System.Xml.XmlUrlResolver.GetEntity(Uri absoluteUri, String role, Type ofObjectToReturn)
    at System.Xml.XmlReader.Create(String inputUri, XmlReaderSettings settings, XmlParserContext inputContext)
    at System.Xml.XmlReader.Create(String inputUri)
    at LotteryApp.RssFeedClass..ctor(String strURL_arg, Int32 intRow_arg, String strLottery_arg) in C:\Projects\ASP_NET\Lottery\Application\MyWebAppProj\LotteryApp\Maintenance\RssFeedClass.vb:line 275
    InnerException: System.IO.IOException
    Message="Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host."
    Source="System"
    StackTrace:
    at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
    at System.Net.FixedSizeReader.ReadPacket(Byte[] buffer, Int32 offset, Int32 count)
    at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult)
    at System.Net.TlsStream.CallProcessAuthentication(Object state)
    at System.Threading.ExecutionContext.runTryCode(Object userData)
    at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
    at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
    at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
    at System.Net.TlsStream.ProcessAuthentication(LazyAsyncResult result)
    at System.Net.TlsStream.Write(Byte[] buffer, Int32 offset, Int32 size)
    at System.Net.PooledStream.Write(Byte[] buffer, Int32 offset, Int32 size)
    at System.Net.ConnectStream.WriteHeaders(Boolean async)
    InnerException: System.Net.Sockets.SocketException
    ErrorCode=10054
    Message="An existing connection was forcibly closed by the remote host"
    NativeErrorCode=10054
    Source="System"
    StackTrace:
    at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
    at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
    InnerException:

    ********
    albanello:
    RayH Hope my above documentation is clear.
    Thanks for your help. I will try any other suggestion you may have. Thanks again
     
  14. RayH

    RayH DiscountASP.NET Lackey DiscountASP.NET Staff

    I hope you understood what I meant when I said I think your XmlReader was trying to process this:

    Code:
    California Lottery Results from SuperPages.com: White Pages, and Online Directories https://www.superpages.com/cities/lottery/lottery-CA.html Thu, 25 Jul 2019 21:55:49 GMThttp://blogs.law.harvard.edu/tech/rss Feed for Node.js https://www.superpages.com/cities/lottery/lottery-CA.html https://www.superpages.com/cities/lottery/lottery-CA.htmlhttps://www.superpages.com/cities/lottery/lottery-CA.html https://www.superpages.com/cities/lottery/lottery-CA.html https://www.superpages.com/cities/lottery/lottery-CA.htmlhttps://www.superpages.com/cities/lottery/lottery-CA.html https://www.superpages.com/cities/lottery/lottery-CA.html https://www.superpages.com/cities/lottery/lottery-CA.htmlhttps://www.superpages.com/cities/lottery/lottery-CA.html https://www.superpages.com/cities/lottery/lottery-CA.html https://www.superpages.com/cities/lottery/lottery-CA.htmlhttps://www.superpages.com/cities/lottery/lottery-CA.html https://www.superpages.com/cities/lottery/lottery-CA.html https://www.superpages.com/cities/lottery/lottery-CA.htmlhttps://www.superpages.com/cities/lottery/lottery-CA.html https://www.superpages.com/cities/lottery/lottery-CA.html
    
    (i.e. the results of a HTTP GET request rather than XML). That's why the link does not work where are the file does (i.e. the link is not in XML format, the file is). A work around may be to download the link as a file instead and then process it. Here is another way to do the same thing.
     
    martino likes this.
  15. RayH:
    "........A work around may be to "download the link as a file instead" and then process it. Here is "another way to do the same thing".

    albanello:
    I chose to use the link you provided "another way to do the same thing" since it is more straight forward with the exact code recommended. The other link you provided "download the link as a file instead" was not as clear to me with all the people throwing their thoughts in.

    The link "another way to do the same thing" provided the code in "C#" so I had to convert it to "Visual Basic" for my needs. I got it working BUT the results were the same as all the other "Readers", we have tried, 5 of the lotteries worked great BUT "California Super Lotto Plus" did not! It threw the same exception all the other "Readers" threw for CSLP.

    Exception->"The underlying connection was closed: An unexpected error occurred on a send."

    The full exception is shown below. I am just guessing but it keeps looking like to me that a setting may not be correct, when the CSLP "WebRequest" is being created!

    First line of code->"my2WebRequest As WebRequest = WebRequest.Create("http://www.superpages.com/lottery/rss/calottery.xml")

    I did not include the actual Visual Basic code, to try and simplify this EMail, but if you want it I will be more than happy to send it.

    Full Exception
    ********
    System.Net.WebException was caught
    Message="The underlying connection was closed: An unexpected error occurred on a send."
    Source="System"
    StackTrace:
    at System.Net.HttpWebRequest.GetResponse()
    at LotteryApp.RssFeedClass..ctor(String strURL_arg, Int32 intRow_arg, String strLottery_arg) in C:\Projects\ASP_NET\Lottery\Application\MyWebAppProj\LotteryApp\Maintenance\RssFeedClass.vb:line 129
    InnerException: System.IO.IOException
    Message="Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host."
    Source="System"
    StackTrace:
    at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
    at System.Net.FixedSizeReader.ReadPacket(Byte[] buffer, Int32 offset, Int32 count)
    at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult)
    at System.Net.TlsStream.CallProcessAuthentication(Object state)
    at System.Threading.ExecutionContext.runTryCode(Object userData)
    at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
    at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
    at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
    at System.Net.TlsStream.ProcessAuthentication(LazyAsyncResult result)
    at System.Net.TlsStream.Write(Byte[] buffer, Int32 offset, Int32 size)
    at System.Net.PooledStream.Write(Byte[] buffer, Int32 offset, Int32 size)
    at System.Net.ConnectStream.WriteHeaders(Boolean async)
    InnerException: System.Net.Sockets.SocketException
    ErrorCode=10054
    Message="An existing connection was forcibly closed by the remote host"
    NativeErrorCode=10054
    Source="System"
    StackTrace:
    at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
    at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
    InnerException:

    ********
     
  16. RayH
    If you press the "Ctrl" key and click on the link provided in the previous EMail, a html page will display, showing the Rss Feed displayed as a Web Page. All other Rss Feeds I have worked with would display the actual Rss Feed Xml file, as "http://www.molottery.com/rss-num.xml" does if you put it in a browser URL.

    Don't know why link not displayed as a link in DASP email like it did in previous email!

    thanks
    Frank Albanello
     

Share This Page