Thursday, January 16, 2014

Solarwinds User Device Tracker: How To: Locate and Auto-notify when Device is on the Network

This is a quick “How To” series on tackling common issues that IT Administrators deal with on a daily basis. This will include a single scenario, and how to use Solarwinds software to address these issues in a few minutes.

Scenario:

 One reoccurring issue that is always on the minds of the majority of Federal Customers is “How do I find rogue devices in my Network?” This used to be a manual process. Based on my personal experience, I was handed a report from my Network Security Officer during an exercise in Germany with a list of 20 unauthorized systems with the System Name and IP Address. This turned into a treasure hunt for finding a single system. I was lucky that day and found the device on the list that was pingable and we drove out to our remote location to pick up the system to hand to our Security Officer. When news hit the entire Brigade within an hour after the Rouge system was confiscated, and for the rest of the month, all Rouge Systems disappeared from the Network.
Not all IT Technicians are lucky enough to find a system easily. With the User Device Tracker Module, it will simplify and automates the process of finding the system.

What information will I need before I begin?

You will need to have one of the following:
MAC Address (Preferred)
IP Address
Hostname


Finding the rouge system:

To find a Rogue system on your Network using UDT, make sure that you have the Hostname, MAC Address, or IP Address. MAC Address is always the best way to search for a system since it is harder to change than an IP or a Hostname. At the Top right, enter in the information, select the ˅ and make sure that the appropriate search type is selected, and then press the Search Icon.

(Image 1)

If UDT finds a single result, it will automatically take you to the Endpoint Details page for the Device.


 (Image 2)

Hot to be automatically notified when the system enters the network. (PART 1)

What if the User Device Tracker is unable to find the device?

 This is where the Device Watch List Resource will be your favorite feature. The Device Watch List will monitor for the device and Alert you when it finds the Device, and you can receive an E-mail once it is found.  On the UDT Summary Page, find the Device Watch List resource and select Manage List.


(Image 3)

In the Manage Watch List Screen, select Add Device.  You will notice when adding the device, it will search for either the MAC Address (Preferred), IP Address, or Hostname. Since I have the MAC, I will enter it in and enter a Name and description of the device in question.


(Image 4)

How to be automatically notified when the system enters the network. (PART 2)

How do I make this an Alert?

Now that the Device in in our Watch List, you will need to go into the Advanced Alert Manager and make sure that the Default Alert “Alert me when watch list item becomes active” is Enabled and enter Trigger Actions to send to Email.
This is what I put into my Email to notify me of the Security Incident. Your E-mail will vary, but this is a good jump off point to include useful information.
*************  Automated Security Incident  ************
Possible Rogue Device has been identified and located via UDT.
**Device Details**
Watch Name: ${WatchName}
Is the Device found on the Network? ${Present}
Time Device found: ${AlertTriggerTime}
Device Notes: ${Note}
**Important URLs**
Device URL: ${WatchListDetailsURL}
Acknowledge Alert URL: ${AcknowledgeURL}
*************  Automated Security Incident  ************

Now when the device is on the Network, I will be E-mailed the location of the device. This is what I now see when the device entered the Network:


 (Image 5)

When I click on the Device URL, it will automatically take me into the Device Tracker Endpoint Details Page (see Image 2). To Acknowledge the Alert, select the Acknowledge URL link and you can notify to the other users that you have been notified and are picking up the system now.


Thursday, January 9, 2014

Solarwinds NetFlow Traffic Analyzer: How to: Group your NetFlow Traffic with IP Address Groups Advanced Reporting

 This is a part of my “How To” series on tackling common issues that IT Administrators deal with on a daily basis. This will include a single scenario, and how to use Solarwinds software to address these issues in a short time.

SCENARIO:

One outstanding feature that is little well known is IP Address Groups in NetFlow. IP Address Groups will allow you to group your IP Address ranges or specific IPs into a group to see what areas of the network are using what percentage of traffic. This is great for seeing what Remote Location is using the Internet Link, but this will also help in understanding your Network Performance as a whole.


WHAT INFORMATION WILL I NEED BEFORE I BEGIN?

Know your IP address Ranges or Specific IP Addresses

Before I begin I will tell you how I setup my network. The Network that I will show is my Internal Lab. IP Address range is fairly simple.
.1-.9 is reserved for Network Devices. DNS and Solarwinds Server happen to fall into this range since this was an earlier existing infrastructure which can be common with older Networks.
.10-.19 – Reserved for Servers
.20-.22 – Special Workstations
.50-.249 is Hosts
.250-.254 is Access Points.
.255 and 255.255.255.255 – Broadcast does not sound necessary, but when you have a broadcast Storm or Multicast issue, this will help later on.
8.8.8.8 and 8.8.4.4 (Google DNS) I also grabbed my Root DNS Servers to monitor DNS Traffic flow.

Adding IP Address Groups

You will need to go into Settings> NTA Settings> Manage IP Address Groups.
Select Add and add in your Ranges and create your Groups. Make sure to enable “ Enable display in Top XX IP Address Groups Resource. “  Here is a sample of what I created.

What am I seeing now:

So what does this give me?

This will give you 2 things
  1. Proper grouping of your Network Devices.
  2. A new view called the IP Address Group Page.

IP Address Group View

This view will get you the same resources the Interface Details page does, but only for the grouped devices. The information will include:
  • IP Address Group Details
  • Top Transmitters
  • Top Receivers
  • Top Applications (Ports)
  • Total Bytes Transferred
  • Top Conversations


When examining my Network, I now use IP Address Groups to see how well the Network is performing. I noticed as I set my groups that my connection to my Root DNS Servers was using more of my Internet Link than I realized. Now I can go into my DNS Configuration and see if there are any Network Improvements I can do to reduce my overall load.

Wednesday, January 8, 2014

Solarwinds: Database Permissions for Solarwinds Orion Products

The Configuration Wizard says I do not have required permissions to the database, how do I fix this?

The Solarwinds Configuration Wizard is reporting this error because the SQL account you have assigned to your Orion installation has not been configured with appropriate permissions to access your Orion database. You must create a new SQL account with required permissions for your Orion installation, as indicated in the following procedure.
Note: To create a new SQL account, you must have access to the following:
  • Microsoft SQL Server Management Studio on the SQL Server hosting your Orion database.
  • sa credentials to the SQL Server hosting your your Orion database.
To create a new SQL account with required permissions for your Orion database:
  1. Using sa credentials, log in to SQL Server Management Studio on the SQL Server hosting your Orion database (Start > All Programs > Microsoft SQL Server > SQL Server Management Studio).
  1. Click + to expand Security > Logins.
  1. Delete all existing SolarWinds Orion database accounts, as follows:
    Warning:You must be logged-in as the sa user. Do not delete the sa user.
  1. Right-click the account name, and then click Delete.
  1. Confirm that the account to be deleted is highlighted, and then click OK.
  1. Click OK to accept the user deletion warning.
  1. Create a new user, as follows:
  1. Right-click Logins, and then select New Login.
  2. Click General in the Select a page pane on the left.
  3. Provide an appropriate Login name.
  4. Select SQL Server authentication.
  1. Enter and confirm an appropriate Password
    Note:
    The password provided must meet at least three of the following criteria:
    • Contains at least one uppercase letter.
    • Contains at least one lowercase letter.
    • Contains at least one number.
    • Contains at least one non-alphanumeric character, e.g., #, %, or ^.
  1. Confirm that Enforce Password Policy is cleared.
  2. In the Default database field, select your Orion database.
    Note: The default Orion database name is NetPerfMon.
  1. Click Server Roles in the Select a page pane on the left.
  2. Confirm that both public and sysadmin are checked.
  1. Click User Mapping in the Select a page pane on the left.
  1. Check your Orion database.
    Note: The default Orion database name is NetPerfMon.
  2. In the corresponding Default Schema field, click Browse (...), enter dbo
  3. Click Check Names, and then click OK.
  4. On the Login - New window, click OK.

  1. To verify the new user, click + to expand Databases > OrionDatabaseName > Security > Users.. Your newly created user should display in the Users list.
    Note: If your newly created user does not display in the Users list, the account you used to initially log in to the SQL Server Management Studio does not have permission to create database accounts.

Solarwinds Installation Resolution when you have SQL installed on the same that you are trying to remove

Correcting the Dependencies when the Customer Removes SQL Server Instance from the Orion Server.

Issue: During the Configuration Wizard after SQL Server has been removed from the Orion Server, you have an Error in the ConfigurationWizard.log stating that it could not find the SQL Service Dependency:
System.InvalidOperationException: Service MSSQL$ was not found on computer '.'. ---> System.ComponentModel.Win32Exception: The specified service does not exist as an installed service

Cause: The Information Services, Collector Serivces and Job Engine v1 and v2 Services will not update and Remove the Dependency of the SQL Server. This can occur if you install the SQL Server Express prior to the Orion Installation, or if you Installed the Evaluation of Orion and it automatically installed SQL Express.

Resolution:
1. Go into Add and Remove Programs, and Remove Information Services, Collector Services and Job Engine v1 and v2
2. Go into the Installers Folder
     a. Server 2003:    C:\Documents and Settings\All Users\Application Data\ Solarwinds\Installers
     b. Server 2008:    C:\ProgramData\Solarwinds\Installers
3. Run the Installer For Information Services, Collector Services and Job Engine v1 and v2 (when prompted select Typical)
4. Run the Configuration Wizard, select Services and select Next.


The Wizard should not complete successfully.

Configuring SNMPv3 for Cisco IOS and ASA devices

    Configuring SNMPv2 is pretty simple. All you need to monitor is SNMP-Server community public RO. SNMPv3 is a whole new beast, but I have taken the pain points out with this document I created to get monitoring setup for your devices. This is for Cisco devices, but I have been told that Brocade and a few other vendors use similar configurations.

    This post will cover

  • Setting up SNMPv3
  • Removing SNMPv3
  • Troubleshooting SNMPv3
  • Setup monitoring for wireless access points
  • Setup monitoring for Solarwinds User Device Tracker or any network topology application
  • Setup SNMPv3 

    This is for Basic setup. If you are looking for a more secure setup, you will need to contact Cisco. This document was only designed to get the device monitored and to troubleshoot any Issues.
    Reference:  SNMPv3

    1. Command: Enable
    2. Command: Config T
    3. Create the View
      1. Command: SNMP-Server view TestSNMPv3View Internet included
      2. ASA Command does not exist, this will default to standard view
    TestSNMPv3View is the View Name
    If you see %Bad OID, then Internet does not exist, use ISO (if exists), or 1.3.6
    1. Included MIB Family is included in the view
    2. Excluded MIB Family is excluded from the view
    1. Create the Group
      1. Command: SNMP-Server group TestSNMPv3Group v3 priv Read TestSNMPv3View Write TestSNMPv3View
      2. Command (ASA Only): SNMP-Server group TestSNMPv3Group v3 priv Read
    TestSNMPv3Group is the Group Name
    1. v1: Group using the v1 security model
    2. v2c: Group using the v2c security model
    3. v3: Group using the User security model (SNMPv3)
    4. Auth: Group using the authNoPriv Security Model
    5. Noauth: Group using the noAuthNoPriv Security Model
    6. Priv: Group using the authPriv Security Model
    7. Access: Specify an access-list associated with this group
    8. Context: Specify a context to associate these views for the group
    9. Notify: Specify a notify view for the Group – Send a syslog every time a view is touched
    10. Read: Specify a read view for the group
    11. Write: Specify a write view for the group
    1. Create a User
      1. Command (same for ASA): SNMP-Server user TestSNMPv3User TestSNMPv3Group v3 auth md5 P@$$w0rd priv DES P@$$w0rd
    TestSNMPv3User is the User Name
    1. Remote: Specify a remote SNMP entity to which the user belongs
    2. v1: Group using the v1 security model
    3. v2c: Group using the v2c security model
    4. v3: Group using the User security model (SNMPv3)
    5. Access: Specify an access-list associated with this group
    6. Auth: Authentication parameters for the user
    7. Encrypted: Specifying passwords as MD5 or SHA digests
    8. MD5: Use HMAC MD5 algorithm for authentication
    9. SHA: Use HMAC SHA algorithm for authentication
    10. 3DES: Use 168 bit 3DES algorithm for encryption
    11. AES: Use AES algorithm for encryption
    12. DES: Use 56 but DES algorithm for encryption

    1. Send to Destination Host (ASA Only)
      1. Command (ASA Only): SNMP-Server Host inside 10.10.1.1 version 3 TestSNMPv3Group
    Note: 10.10.1.1 is the destination host that is able to monitor the Device, if the IP Address of Solarwinds NPM is not in the list, then you will not be able to add the Device
    1.   inside   Name of interface Vlan1
    2.   outside  Name of interface Vlan2

    1. Example of the configuration from start to finish:

    1. Standard Cisco:
    Cisco:enable
    Cisco#config t
    Enter configuration commands, one per line.  End with CNTL/Z.

    Cisco(config)#SNMP-Server view TestSNMPv3View internet included
    Cisco(config)#SNMP-Server group TestSNMPv3Group v3 priv Read TestSNMPv3View Write TestSNMPv3View
    Cisco(config)#SNMP-Server user TestSNMPv3User TestSNMPv3Group v3 auth MD5 P@$$w0rd priv DES P@$$w0rd

    1. Cisco ASA:
    Cisco:enable
    Cisco#config t

    Cisco(config)# SNMP-Server group TestSNMPv3Group v3 priv
    Cisco(config)# SNMP-Server user TestSNMPv3User TestSNMPv3Group v3 auth MD5 P@$$w0rd priv DES P@$$w0rd
    Cisco(config)# SNMP-Server Host inside 10.10.1.1 version 3 TestSNMPv3User
      
    1. Adding the device in Orion:
    Note: Do not initially add Read/Write Credentials, then select Test.

    Removing SNMPv3 Configuration 

    1. Important Commands to use to Remove existing configurations, please use ? for more options:
      1. No snmp-server group
      2. No snmp-server user
      3. No snmp-server host

    Troubleshooting SNMPv3 Configuration

    1. Command: Show snmp view
      1. Views  - contained in groups
        1. Views define what MIBs are available on the Device

    1. The view name we are looking for here is TestSNMPv3View, and you can see it includes everything from Internet down
    2. MIB Iso is 1. and below
    1. Command: Show snmp group 
      1.  Group view associates from the TestSNMPv3Group is the following:
        1. Read view: TestSNMPv3View
        2. Write View: TestSNMPv3View
        3. Security Model: v3 priv

    1. Command: show snmp user

    1. Looking at the User TestSNMPv3User, it is assigned to the group TestSNMPv3Group.

    Troubleshooting an ASA

    Note: Show SNMP View does not work on ASA Devices, you will use def_read_view as the view

    1. Command: Show run | grep SNMP
      1. Shows the current SNMP Configuration (note none is listed, so this is no config)

    1. Shows the current SNMP Configuration. Note that this is the exact same configuration as in step 7, and the password is encrypted.
    2. Also Note the Host and the Interface it is going out on
    SNMPv3 Traps (Orion Core 2011.2 and higher )
    Note: This assumes that you have setup and configured SNMPv3 on the device already.
    1. Add the following while in Configuration Terminal:
      1. Command: snmp-server host 10.10.1.6 version 3 auth TestSNMPv3User version 3
        1. The authentication must match the same as the SNMPv3 configuration
    1. You can add the following on the same command line to generate Traps:
    config syslog aaa_server snmp ( these are basic Trap types sent.)

    Troubleshooting SNMPv3 Traps.

    1. Check the Log File:
      1. Server 2008:
        1.  C:\ProgramData\Solarwinds\Logs\Orion\TrapService.log
      1. Server 2003
        1. C:\Documents and Settings\All Users\Application Data\Solarwinds\Logs\Orion\TrapService.log
    1. If you see the following Error please see This KB
    ERROR TrapService.TrapService - Bad trap packet received from Node with IP <IP of Device>. Error description : Security level is set to 2 but no encryption password was provided.

    View Wireless Information

    Add to your current View:
    1. Command: SNMP-Server view TestSNMPv3View ieee802dot11 included

    Solarwinds User Device Tracker Module, or any Network Topology setup

    While everything works by default on SNMPv2, you will need to add new commands to the Cisco devices to expose per VLAN values for this MIB. According to Cisco, SNMPv2 and SNMPv3 work quite differently when polling the BRIDGE-MIB which contains these layer 2 values. There is no single command that will expose all existing VLANs. If on a certain switch you have devices on VLANs 3, 10, and 41, you needed to add these commands:
    1. Command: snmp-server group OurGroupName v3 priv context vlan-3
    2. Command: snmp-server group OurGroupName v3 priv context vlan-10
    3. Command: snmp-server group OurGroupName v3 priv context vlan-41

1st Post

This page will be covering problems and their resolutions that I have encountered in the IT Sector.

I have been working in IT for over 10 years and have hundreds of documents that I have created to assist customers during my time to resolve all kinds of OS, application, and network issues from my various jobs.

A lot of the information will be Solarwinds software based since I have loads of solutions, tidbits, and tricks to get the most out of network monitoring systems, but I will also cover Virutalization, and a host of issues and resolutions that I have encountered within my Home Lab network.