Django + IPython 0.11 + models

I picked up a useful ipythonrc hack that would load models automatically when executing manage.py shell. Plopping the following in ipy_user_conf.py performed all the model imports and life was great.

This broke with IPython 0.11 which completely changed the config format. IPython added a profile structure but the way Django shell calls IPython, it never seems to load a profile. To get that to work, I just modified the shell command and placed it into addons/management/commands directories as ishell.py, touched __init__.py in all of the subdirectories, and placed ishell.py in the commands directory. 

Now that IPython is changed to an embed call, it inherits the environment from django and it loads the default profile from ipython.   I prefer the subcommand approach because that lets everything be kept with the django setup rather than this personal ipython command.

Relevant commands (Download the original)

import IPython
import datetime

from django.db.models.loading import get_models
for m in get_models():
    exec ("from %s import %s" % (m.__module__, m.__name__))

IPython.embed()

Model Loading via ipy_user.conf.py (IPython < 0.11):

def main():

    load_django_models()

    ip.ex("import datetime")

    ip.ex("zz_curtime = datetime.datetime.now()")

    

# some config helper functions you can use 

def import_all(modules):

    """ Usage: import_all("os sys") """ 

    for m in modules.split():

        ip.ex("from %s import *" % m)

        

def execf(fname):

    """ Execute a file in user namespace """

    ip.ex('execfile("%s")' % os.path.expanduser(fname))



def load_django_models():

    try:

        from django.db.models.loading import get_models

        for m in get_models():

            ip.ex("from %s import %s" % (m.__module__, m.__name__))

        print 'INFO: Loaded Django models.'

    except ImportError:

        pass

pynids on 64-bit linux

Trying to compile pynids gave me fits.  Thankfully I could ask for help ;-).

gcc -pthread -shared build/temp.linux-x86_64-2.5/nidsmodule.o -L../libnids/lib -L../libpcap/lib -lnids -lpcap -o build/lib.linux-x86_64-2.5/nids.so
/usr/bin/ld: /usr/local/lib/libnids.a(libnids.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
/usr/local/lib/libnids.a: could not read symbols: Bad value


1) http://jon.oberheide.org/pynids/ - download
2) cd libnids-1.24
3) cd CFLAGS=-fPIC ./configure --disable-libglib --disable-libnet --disable-shared
4)  python setup.py build

TippingPoint and Blackhole Exploit

TippingPoint DVLabs recently  posted a good analysis of the blackhole exploit kit and the success rates.   Get any group of people together and ask "how many of you have helped someone that had a Fake Antivirus program installed" and you'll have a quick view of how big the problem is.  

The DVLabs post says people using TippingPoint IPS should be well protected against this exploit.   Many of the rules that are providing protection are not enabled by default and/or not 100% effective against this exploit kit.

• CVE-2010-1885   Typo in original post: 9989 - Should be 9899 (B/N)
• CVE-2010-1423   9697 (see below),9698 (B/N)

• CVE-2010-0886   9697 Web Launch Command Injection (B/N)

• CVE-2010-0842   9651 MixerSequencer (Disabled)

• CVE-2010-0840   10985  (Block/Notify)

• CVE-2009-1671   10919  Java Deployment Toolkit ActiveX (Disabled)

• CVE-2009-0927   6255 - Suspicious Javascript (Disabled)

• CVE-2008-2992   6833 - util.printf call (Disabled)

• CVE-2007-5659   6435  Compressed Javascript (Disabled),6436 (Block/Notify)

• CVE-2006-0003   4244 Recommended: Disabled

TippingPoint seems writing rules targetingthe vulnerability and not the specific instance of the exploit.  Even with all of the Java rules turned on, vulnerable JRE versions will still be exploited in the recommended configuration.  It's very difficult to write an IDS rule that detects a malicious .jar or .class file while letting legit applications work.

Currently, I use the following two snort rules to detect Blackhole sites and then use TippingPoint RepDV to block the site.  The actual exploit sites move IP addresses fairly quickly.  I don't believe they show up in RepDV feeds quickly. 



alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"ET CURRENT_EVENTS Potential Blackhole Exploit Pack landing"; flow:established,to_server; content:".php?f="; http_uri; fast_pattern; content:".php?tp="; http_header; pcre:"/\.php\?f=\d+$/U"; classtype:bad-unknown; reference:url,krebsonsecurity.com/2010/10/java-a-gift-to-exploit-pack-makers/; sid:2012688; rev:1;)
 
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"ET TROJAN Potential Blackhole Exploit Pack Binary Load Request"; flow:established,to_server; content:".php?f="; nocase; http_uri; content:"&e="; nocase; http_uri; content:!"Referer|3a|"; http_header; content:"User-Agent|3a|"; nocase; http_header; content:"Host|3a|"; nocase; http_header; distance:0; pcre:"/\.php\?f=\d+&e=\d+$/U"; classtype:bad-unknown; reference:url,krebsonsecurity.com/2010/10/java-a-gift-to-exploit-pack-makers/; sid:2012169; rev:6;)

Within the 2012169 rule, you'll see the JRE version used to download the exploit.  If the version is less than 1.6.0.24, you can assume it was successful and start investigating.   Converting these into a CSW would be useful or for DVLabs to implement similar.   I'd love to have TippingPoint providing timely for exploit kits as well as post-infection indicators for the popular malware.

Exporting Config (permit tcp 10.26.2.6:any -> 10.26.2.1:4001)

Reading this is almost impossible. I know it's not intended to be human readable but this is a little silly.

  <?xml version="1.0" encoding="UTF-8" ?>
<fpc4:Root xmlns:fpc4="http://schemas.microsoft.com/isa/config-4" xmlns:dt="urn:schemas-microsoft-com:datatypes" StorageName="FPC" StorageType="0">
  <fpc4:Build dt:dt="string">7.0.7734.100</fpc4:Build>
  <fpc4:Comment dt:dt="string" />
  <fpc4:Edition dt:dt="int">32</fpc4:Edition>
  <fpc4:EnterpriseLevel dt:dt="int">2</fpc4:EnterpriseLevel>
  <fpc4:ExportItemClassCLSID dt:dt="string">{59740B3A-8771-492C-AF59-7764F4F939EF}</fpc4:ExportItemClassCLSID>
  <fpc4:ExportItemCompatibilityVersion dt:dt="int">2</fpc4:ExportItemCompatibilityVersion>
  <fpc4:ExportItemScope dt:dt="int">0</fpc4:ExportItemScope>
  <fpc4:ExportItemStorageName dt:dt="string">{6248D492-DFBA-4F65-8E28-25A3595A1F4F}</fpc4:ExportItemStorageName>
  <fpc4:IsaXmlVersion dt:dt="string">7.3</fpc4:IsaXmlVersion>
  <fpc4:OptionalData dt:dt="int">12</fpc4:OptionalData>
  <fpc4:Upgrade dt:dt="boolean">0</fpc4:Upgrade>
  <fpc4:ConfigurationMode dt:dt="int">0</fpc4:ConfigurationMode>
<fpc4:Arrays StorageName="Arrays" StorageType="0">
<fpc4:Array StorageName="{F4287363-58CC-47DE-B79C-7FAC6B6ACCE2}" StorageType="0">
  <fpc4:AdminMajorVersion dt:dt="int">0</fpc4:AdminMajorVersion>
  <fpc4:AdminMinorVersion dt:dt="int">0</fpc4:AdminMinorVersion>
  <fpc4:Components dt:dt="int">-1</fpc4:Components>
  <fpc4:DNSName dt:dt="string" />
  <fpc4:Name dt:dt="string" />
  <fpc4:Version dt:dt="string">0</fpc4:Version>
<fpc4:ArrayPolicy StorageName="ArrayPolicy" StorageType="0">
  <fpc4:Name dt:dt="string" />
<fpc4:PolicyRules StorageName="PolicyRules" StorageType="0">
<fpc4:PolicyRule StorageName="{6248D492-DFBA-4F65-8E28-25A3595A1F4F}" StorageType="1">
  <fpc4:Enabled dt:dt="boolean">1</fpc4:Enabled>
  <fpc4:Name dt:dt="string">Permit Access to the MS Certificate Authority</fpc4:Name>
  <fpc4:Order dt:dt="bin.hex">f6fbffff0100000040a4836ebaa6cb01</fpc4:Order>
<fpc4:SelectionIPs StorageName="SourceSelectionIPs" StorageType="1">
  <fpc4:Refs StorageName="Networks" StorageType="1" />
  <fpc4:Refs StorageName="NetworkSets" StorageType="1" />
<fpc4:Refs StorageName="Computers" StorageType="1">
<fpc4:Ref StorageName="{C98C9EF0-1D34-4E35-A260-C54C6B865B3F}" StorageType="1">
  <fpc4:Name dt:dt="string">{BD2681AD-0B98-4796-A6F5-FA4340AA9FB3}</fpc4:Name>
  <fpc4:RefClass dt:dt="string">msFPCComputer</fpc4:RefClass>
  </fpc4:Ref>
  </fpc4:Refs>
  <fpc4:Refs StorageName="AddressRanges" StorageType="1" />
  <fpc4:Refs StorageName="Subnets" StorageType="1" />
  <fpc4:Refs StorageName="ComputerSets" StorageType="1" />
  <fpc4:Refs StorageName="EnterpriseNetworks" StorageType="1" />
  </fpc4:SelectionIPs>
<fpc4:AccessProperties StorageName="AccessProperties" StorageType="1">
  <fpc4:ProtocolSelectionMethod dt:dt="int">1</fpc4:ProtocolSelectionMethod>
<fpc4:SelectionIPs StorageName="DestinationSelectionIPs" StorageType="1">
  <fpc4:Refs StorageName="Networks" StorageType="1" />
  <fpc4:Refs StorageName="NetworkSets" StorageType="1" />
<fpc4:Refs StorageName="Computers" StorageType="1">
<fpc4:Ref StorageName="{A101F1C0-B478-4BD7-96C7-550E453AE803}" StorageType="1">
  <fpc4:Name dt:dt="string">{3B4CFD5E-20EF-41CC-9BE2-C4C0CC1298C2}</fpc4:Name>
  <fpc4:RefClass dt:dt="string">msFPCComputer</fpc4:RefClass>
  </fpc4:Ref>
  </fpc4:Refs>
  <fpc4:Refs StorageName="AddressRanges" StorageType="1" />
  <fpc4:Refs StorageName="Subnets" StorageType="1" />
  <fpc4:Refs StorageName="ComputerSets" StorageType="1" />
  <fpc4:Refs StorageName="EnterpriseNetworks" StorageType="1" />
  </fpc4:SelectionIPs>
  <fpc4:Refs StorageName="DestinationDomainNameSets" StorageType="1" />
<fpc4:Refs StorageName="ProtocolsUsed" StorageType="1">
<fpc4:Ref StorageName="{546B9105-BC04-49BE-A87B-F04771E754A4}" StorageType="1">
  <fpc4:Name dt:dt="string">{5265E2A4-781E-4032-BF14-429A5FF89907}</fpc4:Name>
  <fpc4:RefClass dt:dt="string">msFPCProtocol</fpc4:RefClass>
  </fpc4:Ref>
  </fpc4:Refs>
  <fpc4:Refs StorageName="ContentTypeSetsUsed" StorageType="1" />
  <fpc4:Refs StorageName="URLSet" StorageType="1" />
<fpc4:Refs StorageName="UserSets" StorageType="1">
<fpc4:Ref StorageName="{501E71D9-20DD-4354-A35E-A063AA55D40B}" StorageType="1">
  <fpc4:Name dt:dt="string">{DFFB7833-9365-4184-AABC-7CAFB018A7FA}</fpc4:Name>
  <fpc4:RefClass dt:dt="string">msFPCUserSet</fpc4:RefClass>
  </fpc4:Ref>
  </fpc4:Refs>
  <fpc4:Refs StorageName="UrlCategory" StorageType="1" />
  <fpc4:Refs StorageName="UrlCategorySet" StorageType="1" />
  </fpc4:AccessProperties>
  <fpc4:MalwareInspectionProperties StorageName="MalwareInspectionProperties" StorageType="1" />
<fpc4:Ref StorageName="PolicyGroups" StorageType="1">
  <fpc4:RefClass dt:dt="string">msFPCPolicyGroup</fpc4:RefClass>
  </fpc4:Ref>
  </fpc4:PolicyRule>
  </fpc4:PolicyRules>
  </fpc4:ArrayPolicy>
<fpc4:RuleElements StorageName="RuleElements" StorageType="0">
<fpc4:Computers StorageName="Computers" StorageType="0">
<fpc4:Computer StorageName="{BD2681AD-0B98-4796-A6F5-FA4340AA9FB3}" StorageType="2">
  <fpc4:Description dt:dt="string">Internal Interface for TMG</fpc4:Description>
  <fpc4:IPAddress dt:dt="string">10.26.2.6</fpc4:IPAddress>
  <fpc4:Name dt:dt="string">PublishingRule::Server#002</fpc4:Name>
  </fpc4:Computer>
<fpc4:Computer StorageName="{3B4CFD5E-20EF-41CC-9BE2-C4C0CC1298C2}" StorageType="2">
  <fpc4:Description dt:dt="string">Master RPC Server for CA and AD Controller</fpc4:Description>
  <fpc4:IPAddress dt:dt="string">10.26.2.1</fpc4:IPAddress>
  <fpc4:Name dt:dt="string">AD Controller</fpc4:Name>
  </fpc4:Computer>
  </fpc4:Computers>
<fpc4:Protocols StorageName="Protocols" StorageType="0">
<fpc4:Protocol StorageName="{5265E2A4-781E-4032-BF14-429A5FF89907}" StorageType="2">
  <fpc4:Components dt:dt="int">-5</fpc4:Components>
  <fpc4:Name dt:dt="string">Certificate Services on CONAD</fpc4:Name>
  <fpc4:Predefined dt:dt="boolean">0</fpc4:Predefined>
  <fpc4:ProtocolCategory dt:dt="int">1</fpc4:ProtocolCategory>
  <fpc4:ProtocolConnections StorageName="SecondaryConnections" StorageType="2" />
  <fpc4:Refs StorageName="ApplicationFilters" StorageType="2" />
<fpc4:ProtocolConnections StorageName="PrimaryConnections" StorageType="2">
<fpc4:ProtocolConnection StorageName="{8409561C-A81F-4518-B7B0-310BA74FCCF2}" StorageType="2">
  <fpc4:Direction dt:dt="int">1</fpc4:Direction>
  <fpc4:PortHigh dt:dt="int">4001</fpc4:PortHigh>
  <fpc4:PortLow dt:dt="int">4001</fpc4:PortLow>
  </fpc4:ProtocolConnection>
  </fpc4:ProtocolConnections>
  <fpc4:Ref StorageName="AssociatedStandardProtocol" StorageType="2" />
  </fpc4:Protocol>
  </fpc4:Protocols>
<fpc4:UserSets StorageName="User-Sets" StorageType="0">
<fpc4:UserSet StorageName="{DFFB7833-9365-4184-AABC-7CAFB018A7FA}" StorageType="2">
  <fpc4:Description dt:dt="string">Predefined user set representing all users. A rule defined using this set will apply to all users, both authenticated and unauthenticated.</fpc4:Description>
  <fpc4:Name dt:dt="string">All Users</fpc4:Name>
  <fpc4:Predefined dt:dt="boolean">1</fpc4:Predefined>
  <fpc4:Accounts StorageName="Access" StorageType="2" />
  <fpc4:NonWindowsUsers StorageName="NonWindowsUsers" StorageType="2" />
  </fpc4:UserSet>
  </fpc4:UserSets>
  </fpc4:RuleElements>
  </fpc4:Array>
  </fpc4:Arrays>
  </fpc4:Root>

UAG 2010 + AD + Microsoft Certificate Authority Services

I'm working in the test lab to play with UAG and see what problems it solves and creates for me at work.  My test lab is fairly simple set of systems running Server 2008 R2.

• 1 Server running AD Controller running default Certificate Services
• 1 Server running ADFS
• 1 Server running SharePoint 2010
• 1 Server running UAG 2010

My goal is to play with many different ways of publishing sites off UAG by creating multiple UAG trunks with different authentication parameters.   A simple requirement was that I wanted to make sure that UAG could easily add new SSL certs from the local domain certificate authority.  My symptom was UAG could not self-register SSL certificate requests. 

> certutil -ping -config CONUAB-AD.CONAD.CONUAB.COM\CONAD-CONUAB-AD-CA

Server could not be reached:  THe RPC server is unavailable. 0x800706ba (WIN32: 1722)

In trying to resolve this command,  I ran wireshark and noticed a number of strange "DCERPC BInd_ack: call_id: 3 Unknown Result(3)" errors coming from my DC/CA system.  Even stranger, wireshark could see none of the traffic to the DC/CA.   Running wireshark from the DC, I then found out I was getting requests but they were coming through as malformed per the packet analysis.   Looking a legit request from my ADFS server, it appeared that those "invalid" DCEPRC had something to do with how the certificate enrollment process worked.

Searching TMG and UAG and recalling that one of the key features of ISA was to filter RPC for exchange, I then started looking for ways to see this being blocked.  Going to Forefront TMG Management -> Logs & Reports showed a good amount of "FWX_E_CONNECTION_KILLED" for 10.26.2.6 (UAG Inside) -> 10.26.2.1 (DC/CA).   This led me to a technet blog to disable RPC compliance.

TMG Management -> Firewall Policy -> Tasks -> Edit System Policy -> Active Directory -> “Enforce Strict RPC” -> OFF

 Following the articles (1,2,3), what I needed to do was now verify that the firewall was blocking my access and create a firewall rule to permit the UAG box to talk to CA services.   To let it be a "tight" firewall rule, you need to force the CA server to bind to a single port.

On the AD/CA server (disable RPC for CA + bind to a port)

1. certutil -setreg ca\interfaceflag +0x8  
2. dcomcnfg -> Components -> DCOM Config -> CertServ -> Use Static End Point 4001
3. net stop  certsvc
4. net start certsvc
5. certutil -ping (from the CA) to verify things are working again

On the UAG Server:

1. Opened Forefront TMG Management
2. Firewall Policy -> Tasks -> Create Access Rule
3. Named "Permit Access to internal CA"
4. Protocols -> New -> TCP/4001 start/4001 end named "Local Certificate Services"
5. From:  UAG Internal (New Computer) that was my 10.26.2.6 IP.
6. To:  "AD Controller" since my CA was running in AD main box. 
7. Applied rules 

> certutil -ping -config CONAD-AD.CONAD.CONUAB.COM\CONAD-CONUAB-AD-CA CertUTil: -ping command completed successfully!

From the outside, this was much more difficult than I had hoped since I had to dive into TMG to figure out prepwork for UAG.  I did learn a lot more about how CA works.  If there's the support for binding things to a single port, why is that not used more often so it works with firewalls easier?

Update:   Using the UAG generated names caused UAG to not be able to publish rules.  "Firewall settings could not be confgured."  Regenerated a new name for 10.26.2.6

Keeping Smaller Nessus Files

I've been playing a lot with Nessus XML lately and doing authenticated scans and assigning out result responsibility. One issue I've run into with the nessus gui, is each time you save your .nessus file, it constantly grows in size.

Here's a realy simple tool to chop out the most recent nessus scans. I use the lxml variant of ElementTree but there's no lxml exclusive functionality used. lxml is just much faster and supports xpath which I've used in my result importer.

import sys,os
import unittest
from lxml.etree import ElementTree

if __name__=='__main__':
   import argparse
   parser = argparse.ArgumentParser()
   parser.add_argument('-f', action='store_true', help='force output overwrite', dest='force')
   parser.add_argument('input', help='input filename to extract from', nargs=1)
   parser.add_argument('output', help='output filename to extract from', nargs=1)
   opts = parser.parse_args(sys.argv[1:])

  
   et = ElementTree()
   et.parse(opts.input[0])

   # Get the root and remove all but the last report
   root = et.getroot()
   for rpt in root.findall("Report")[:-1]:
       root.remove(rpt)

   if not opts.force and os.path.exists(opts.output[0]):
       print >>sys.stderr, "File Exists!, exiting"
       sys.exit(1)

   et.write(opts.output[0])

Outlook 2010 + Social Connector + Security Zones

I've not seen many people explore Outlook 2010 and Internet Zones yet. I have a restricted Internet Security Zone so I can run a vulnerable version of Java for testing against specific sites on my main Windows 7 workstation. When loading Outlook 2010, I did find it useful to see a conversation history of people. The trouble was, I got useful information and then an annoying error message saying "check security settings" with no specifics.

I run in High Security for the Internet Zone so I was sure that the error was related to some component of Outlook hitting the Internet Zone. The trouble was finding the content and how it was being rendered. After looking for a bit, I had a dumb idea: Check IE History. Sure enough, I saw plenty of %AppData%\Local\Temp\{GUID}.html files in temp. My default cygwin less warned that it was a binary file. Saving unicode handling errors for another day, I used notepad to view that it was requesting res://C:\\Program Files (x86)\\Microsoft Office\\Office14\\1033\\SocialConnectorRes.dll/GIF/#290.

I then loaded the HTML file in IE directly. As far as I can tell, the entire point of loading this page is to pull social network conversations and display a pretty loading page. Still not positive what this connector DLL is doing tbut I hope this helps point someone in the right direction.