Troubleshooting ASP.NET

Hello again and welcome to the November '04 edition of the Support Voice Column. I would like to thank Jim Cheshire , a support engineer here at Microsoft supporting ASP .NET, for his contributions. Jim has great ideas for the Support Voice column and wanted to share them. Look for Jim's contributions over the next few months, and as always, send us your suggestions for future columns. Thank you Jim!

Jim has been working with Microsoft for six years on the FrontPage, VB, and ASP .NET teams. During that time, he has written for the Office Developer Center on MSDN, and he is the author of a book on FrontPage, Special Edition Using Microsoft Office FrontPage 2003. Jim also has a Web site where he provides free add-ins for FrontPage to enable Web developers to make the most out of Microsoft products. Here is that Web site address:So please pull up a chair, kick the shoes off, and read through our column all about troubleshooting ASP.NET , and remember, you can submit your ideas to us using the "ASK FOR IT" link included in every column we publish.

Jeremy

Have you seen the movie Shrek? The star of the movie, Shrek, is an ogre, an ugly green creature with awful hygiene habits. Early in the movie, Shrek proclaims the complexity of ogres by saying that �ogres are like onions. They have layers.�

ASP.NET is much like an ogre. It has many layers -- the Web client, any intermediary device like proxy servers, the Web server and the network that it�s on, and any other resources the application touches. All of those layers make ASP.NET difficult to troubleshoot unless you know which tool to use and how to properly use that tool.

At the risk of making myself obsolete, this series will teach you how to use some of the tools we use in PSS to resolve issues. I�ll go over how to use some really cool tools such as:

• Network Monitor
• Fiddler
• Filemon / Regmon
• Debugging Tools for Windows (Windbg)
• SOS (a Windbg extension for debugging managed code)
• DBGClr

I won�t go into the intimate details of using these tools, but I will provide you with a solid foundation on using these tools so that you can drill down into the root cause of many problems that you might encounter.

Sniffers

The first part of this series will deal with network sniffers. Let�s dive right in.

Example problem:

You have an ASP.NET application that uses Windows-integrated authentication and has �anonymous� turned off. One of your users is reporting that they are not able to access the site. Instead, they are repeatedly prompted for a user name and password. You believe that the permissions are correct for this user.

This kind of issue is the perfect candidate for troubleshooting using a network sniffer. A sniffer will let you determine whether or not the client is sending authentication information to the server. The network sniffer we use at Microsoft is Network Monitor.

Network Monitor

You can download a time-bombed version of Network Monitor from the following Microsoft FTP site: The password on this Zip file is trace. After you install Network Monitor, you will find it by clicking Administrative Tools, and then clicking Network Analysis Tools.

Selecting the network interface to capture

When you start Network Monitor for the first time, the first thing you must do is select the network card that you want to use in the dialog box shown in Figure 1 .

Figure 1: Choosing a network in Network Monitor

Note that in Figure 1, the selected interface is a dial-up connection or VPN adapter as indicated in the properties for the interface. In most cases, you will want to select one of the interfaces other than the dial-up or VPN connection. I chose to use a screenshot of the VPN adapter that you don�t want to choose because I wanted you to clearly see how it would be identified. Once again, in almost all cases, the adapter that you want to choose is the one that doesn�t look like the one pictured in Figure 1.

If you need to change the network card that you are capturing, you can access the dialog box to do so by clicking the Networks option on the Capture menu.

Setting the buffer size

Network Monitor has a default capture buffer of 1 MB. That means that after it collects 1 MB of network data, it starts to overwrite the trace. In many cases, you may want to increase that buffer. To do that, click Capture and then click Buffer Settings. This will display the Capture Buffer Settings dialog box where you can increase the size of the buffer. The size that you want to specify depends on how much network traffic you are seeing on your network. For an issue where you intend to generate the frames you are interested in immediately upon starting the capture, a 2- or 3-MB buffer should be more than sufficient.

You�ll notice that in the Capture Buffer Settings dialog box, you can also change the frame size. This is useful in cases where you want to capture just the headers that are sent. By decreasing the frame size, you can save some space in the buffer and still capture the headers you need. I won�t go into detail on using that in this article.

Starting the capture

Now that you�ve got your buffer set, you�re ready to start capturing data. You can start capturing in several different ways:

• Press F10 on the keyboard.
• Click Capture, and then click Start from the menu.
• Click the Start Capture button on the toolbar (this button looks like a play button).

As you capture packets with Network Monitor, you�ll see the meters moving and the statistics changing, as shown in Figure 2. If you don�t see anything happening when you are capturing, you likely need to change the network card you are capturing.

Figure 2: Network Monitor while capturing packets

After you�ve started capturing in Network Monitor, reproduce the issue that you need to capture, and then stop the capture by clicking the Stop button in Network Monitor, clicking Capture, and then clicking Stop, or by pressing F11 on your keyboard. You are now ready to examine the data that was captured.

Note Network Monitor captures only data that goes out over the wire. Therefore, you usually cannot capture a request against your Web application by browsing at the console. In some cases, browsing using your IP address or fully qualified domain name will allow you to capture on the local computer.

Examining captured data

After you�ve stopped the capture, you can view the captured data by clicking Capture, and then clicking Display Captured Data, or by pressing F12 on your keyboard. By default, you see all data that went across the wire while you were capturing as shown in Figure 3.

Figure 3: Captured data in Network Monitor

In this case, it would be beneficial to be able to just view the HTTP packets. That�s easily accomplished by filtering the captured data to display only the HTTP protocol. To filter the captured data, click Capture, and then click Filter, or press F8 on your keyboard, or click the toolbar button that looks like a funnel.

In the Display Filter dialog box, double-click the Protocol == Any filter to display the Expression dialog box as shown in Figure 4.

Figure 4: The Expression dialog box provides powerful filtering capabilities

By default, Network Monitor shows all protocols. To show only the HTTP protocol, click Disable All, double-click the HTTP protocol to select it, and then click OK. Network Monitor will display only the HTTP protocol.

Note You can filter Network Monitor before capturing so that it only captures frames that match the filter you set. In most cases, I prefer to filter after the capture.

After filtering the captured packets so that only the HTTP protocol is displayed, Network Monitor will display each frame of the captured HTTP data. By double-clicking a frame, you can review detailed information on that packet. In the scenario we�re dealing with, we are trying to determine if the client has sent any authentication information to the Web server. The capture shown in Figure 5 shows the response from the GET request for the ASP.NET page. Notice that the Web server has responded back with a 401 status code indicating that access has been denied to the Web page.

Figure 5: Network Monitor showing an �Access Denied� response when authentication is required

By looking at the WWW-Authenticate headers, you can see in Figure 5 that the Web server accepts both Kerberos (Negotiate) and NTLM authentication. Therefore, we expect that the client will send authentication information automatically after the �Access Denied� message we see here.

Sequence Numbers

To find the client�s response to the packet shown in Figure 5, we need to look at the sequence numbers in this response. The concept of sequence numbers is frequently misunderstood in a review of Network Monitor traces. They are the key to understanding the order in which things occurred.

In the middle pane in Figure 5, you�ll notice that the HTTP protocol has been expanded to reveal all of the HTTP headers. Just above the HTTP protocol is the TCP protocol, and you can see the sequence numbers and the ack number as part of the TCP segment. The sequence numbers (represented by seq in the TCP packet) provide a way to identify the specific TCP segment. Each TCP sequence should be accompanied by an acknowledgement, or ack, of that sequence.

Here�s a snippet from the trace of an HTTP GET request: Note that the ending sequence number is 3410290882. Therefore, to find the server�s response, you must find the frame that acknowledges that sequence. In other words, you are looking for a response with an ack of 3410290882. Here is that frame:These numbers are important because there are times when the response will appear before the request in a Network Monitor trace. The only way to be certain of the order of packets is to check the segments using these numbers.

Since we are looking over this trace to see if the client is sending authentication information, we can use the TCP segments to track the HTTP GET requests and the response from the server. Here is a snippet from the frame that sends authentication information from the client:What does this tell us? We can see that the Authorization header is set to "Negotiate" and we can see a long string of characters sent in that header. This response tells us that the client and the server are negotiating an NTLM connection. We know that NTLM authentication is being used here because the first character is a '"T." If it was a "Y," it would be Kerberos. The header is set to "Negotiate" instead of "NTLM." This does not mean it will use Kerberos or NTLM, but that it will "Negotiate" the authorization method and try Kerberos first if it is able. If it cannot use Kerberos, it will use NTLM.

Figure 6 is a screen shot of the frame shown above so that you can see where the authentication information is sent. Note that the HTTP headers in the middle frame have been expanded so that we can see the details of each header.
Figure 6: The frame containing authentication information

This frame tells us that the client is most definitely sending authentication information. If the user gets denied access at this point, it is either because the user does not have permission to the resource or it is because the authentication information is being changed at some point across the wire.

Another way to filter

In many cases, you may be troubleshooting a server that has a lot of traffic. In those cases, you might see a large number of HTTP frames and it may be difficult to find the right ones. By using more filtering expressions, you can easily get right to the correct frames.

Click the Filter button, or press F8 on your keyboard, to bring up the Display Filter dialog box. On the right side, click Expression to add a new expression. There are two different filters you might want to use in this scenario: filter by IP address and filter by HTTP status code.

To filter by IP address, click the Property tab, and scroll down in the left pane until you see IP. Expand IP, and scroll down until you see Source Address. Select == for the relation, and enter the IP address of the client computer as shown in Figure 7. After you do that, you will only see frames coming from the IP address you entered.

Figure 7: Filtering by IP Address

It�s important to note that many times the IP address may not be the best property on which to filter a trace. Oftentimes users will be hitting a Web server via a router or a firewall. In those cases, the IP address will often be the IP address of the router or firewall and not the client.

You can also filter by HTTP status code. If you�d like to quickly locate the frame where a 401 is sent to the client, you can filter on the 401 HTTP status code. To do that, select HTTP in the Protocol:Property list, and expand it. Scroll down, and then click Status Code. Select == for the relation, click the Decimal radio button (important), enter 401 in the Value field as shown in Figure 8, and then click OK. Network Monitor will display only those frames where a 401 status code was sent, as shown in Figure 8.

Figure 8: Filtering by HTTP status code
After you�ve located the frame where the 401 was sent, you can then press F7 on your keyboard to turn off the filter. The frame where the 401 was sent will still be highlighted, and you can then examine the surrounding frames for the client�s response to the 401.

Fiddler � The HTTP Debugging Proxy

If you are browsing from Internet Explorer on the client computer that you are troubleshooting, you can use Fiddler (shown in Figure 9) instead of Network Monitor to capture the HTTP information sent from the client. Fiddler captures only HTTP packets and integrates directly into Internet Explorer.

Figure 9: Fiddler

In Figure 9, you can see the 401 response from the server and then you can see the client respond with a Negotiate header (highlighted). In this case, a runtime error occurs after the client sends authentication (which is unrelated to this issue), but we can clearly see that the client did its part by sending the Kerberos ticket.

You can download Fiddler from the following Web site:

What�s next?

Using Network Monitor or Fiddler, you can often rule out the client as being a problem in authentication failures. Next month, we�ll go over how to use Filemon and Regmon from SysInternals. These tools are especially helpful in troubleshooting permission problems on the file system and in the registry.

As always, feel free to submit ideas on topics you want addressed in future columns or in the Knowledge Base using the Ask For It form.