«

»

Jan 12 2013

Visualizing Dionaea’s results with DionaeaFR

Hello readers and honeypot enthusiasts. As I was writing a couple of articles on basic malware analysis, I noticed today that a new visualization tool was released for Dionaea malware honeypot! In fact I had in mind to develop something along the lines of Kippo-Graph for Dioanea as well, so I am very happy to have stumble upon it (mostly by accident).

The tool is called DionaeaFR and I’ve found it really helpful in the analysis phase of a honeypot’s activity. It provides a general overview of the malicious connections but it can also zoom in on individual attacks. The fact that’s an aesthetically pleasing utility is also a big plus on my book. The only downside I’ve found is that it couldn’t process a rather large database I had (around 500mb, which btw is normal for Dionaea). The webserver it deploys was being killed after a while, but this could be due to quite low VPS specs.

DionaeaFR is written in Python, uses the Django framework and a number of other libraries, mostly client-side JS. It is maintained by Ruben Espadas. Let me guide you through its installation procedure. It is presumed that you already have Dionaea installed, using its installation guide.

1) Install pip (Python package manager) and python-netaddr package:

apt-get install python-pip python-netaddr

2) Continue with the prerequisites using pip for automated installation:

pip install Django
pip install pygeoip
pip install django-pagination
pip install django-tables2
pip install django-compressor
pip install django-htmlmin

3) Get and install django-tables2-simplefilter manually:

cd /opt/
wget https://github.com/benjiec/django-tables2-simplefilter/archive/master.zip -O django-tables2-simplefilter.zip
unzip django-tables2-simplefilter.zip
mv django-tables2-simplefilter-master/ django-tables2-simplefilter/
cd django-tables2-simplefilter/
python setup.py install

4) Download and install PySubnetTree:

cd /opt/
git clone https://github.com/bro/pysubnettree.git
cd pysubnettree/
python setup.py install

5) Compile and install Node.js from sources:

cd /opt/
wget http://nodejs.org/dist/v0.8.16/node-v0.8.16.tar.gz
tar xzvf node-v0.8.16.tar.gz
cd node-v0.8.16
./configure
make
make install

6) Install LESS and promise using npm (Node.js package manager):

npm install -g less
npm install -g promise

7) Download DionaeaFR itself:

cd /opt/
wget https://github.com/RootingPuntoEs/DionaeaFR/archive/master.zip -O DionaeaFR.zip
unzip DionaeaFR.zip
mv DionaeaFR-master/ DionaeaFR

8) Get Maxmind’s GeoIP and GeoLite databases for DionaeaFR:

cd /opt/
wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
gunzip GeoLiteCity.dat.gz
gunzip GeoIP.dat.gz
mv GeoIP.dat DionaeaFR/DionaeaFR/static
mv GeoLiteCity.dat DionaeaFR/DionaeaFR/static

9) Copy and edit the sample settings file:

cp /opt/DionaeaFR/DionaeaFR/settings.py.dist /opt/DionaeaFR/DionaeaFR/settings.py
nano /opt/DionaeaFR/DionaeaFR/settings.py

There you’ll want to change line 18 that points to Dionaea’s SQLite db.

10) We are ready to start the webserver:

mkdir /var/run/dionaeafr #for DionaeaFR's pid file
cd /opt/DionaeaFR/
python manage.py collectstatic #type yes when asked
python manage.py runserver 0.0.0.0:8000

The interface is now accessible through: http://SERVER-REMOTE-IP:8000
Let’s take a closer look to a small dataset created after four hours on an low-end VPS…

45 comments

9 pings

Skip to comment form

  1. Thanassis

    Nice guide, I have a problem with the attackers page where I get an exception

    Exception Type: KeyError at /maps/attackers/
    Exception Value: ‘latitude’

    Any ideas?

    1. Ion

      Hi Thanassis, are you using HoneyDrive for this or not?

  2. Marc

    Thanks mate, got this setup running now, and its already dealing with attacks from asia 🙂

    1. Ion

      Great! 🙂 Care to share some results perhaps?
      Regards, Ion.

  3. Katerina

    Hello,

    At HoneyDrive I try to use DionaeaFR and I get this error…

    ………
    Exception Value:
    [Errno 13] Permission denied: ‘/opt/dionaeaFR/CACHE/css/styles.7ef9abf696d5.css’
    Exception Location: /usr/local/lib/python2.7/dist-packages/django/core/files/storage.py in delete, line 224
    Python Executable: /usr/bin/python
    Python Version: 2.7.3

    …………

    Why this is happening? What must I do? Any idea?
    Thanks!!!

    1. Ion

      Hello Katerina,
      can you try step 10 again from the post above and paste the output here?

  4. Katerina

    If I set debug to false in settings.py, I get the following in command line

    ss HTTP/1.1″ 404 1068

    [18/Sep/2013 19:17:09] “GET /static/css/bootstrap-responsive.min.css HTTP/1.1” 404 1068
    [18/Sep/2013 19:17:10] “GET /static/js/bootstrap.min.js HTTP/1.1” 404 1068
    [18/Sep/2013 19:17:28] “GET /maps/countries HTTP/1.1” 301 0
    [18/Sep/2013 19:17:29] “GET /maps/countries/ HTTP/1.1” 200 1823
    [18/Sep/2013 19:17:30] “GET /static/css/bootstrap.min.css HTTP/1.1” 404 1068
    [18/Sep/2013 19:17:30] “GET /static/css/bootstrap-responsive.min.css HTTP/1.1” 404 1068
    [18/Sep/2013 19:17:30] “GET /static/css/styles.less HTTP/1.1” 404 1068
    [18/Sep/2013 19:17:30] “GET /static/css/jquery-jvectormap-1.0.css HTTP/1.1” 404 1068
    [18/Sep/2013 19:17:30] “GET /static/django_tables2/themes/bootstrap/css/screen.css HTTP/1.1” 404 1068
    [18/Sep/2013 19:17:30] “GET /static/js/less-1.3.1.min.js HTTP/1.1” 404 1068
    [18/Sep/2013 19:17:30] “GET /static/js/jquery-jvectormap-1.0.min.js HTTP/1.1” 404 1068
    [18/Sep/2013 19:17:30] “GET /static/js/jquery-1.7.2.min.js HTTP/1.1” 404 1068
    [18/Sep/2013 19:17:30] “GET /static/js/jquery-jvectormap-world-mill-en.js HTTP/1.1” 404 1068

    So I think css is not working correctly… but again I cannot see how I can solve this..

    Thank you!!!!

    1. Ion

      Hello Katerina, I think you forgot to run the “python manage.py collectstatic” command. Can you verify this? Then you start the server with “python manage.py runserver 0.0.0.0:8000”. Regards, Ion

    2. Ion

      Hello Katerina. Please start the server as root (sudo python manage.py runserver 0.0.0.0:8000) and try again. Also, don’t forget to run the collectstatic command before that. Regards, Ion.

  5. katerina

    Again, tha same..
    I reinstalled it but.. still the same problem…

  6. jamie

    I had to
    a. get python 2.7
    b. pip install netaddr
    as well.

    but nice guide, thank you!

  7. Koen

    On Ubuntu I had to add

    apt-get install build-essential
    apt-get install python-dev
    apt-get install git
    pip install django-filter

    and do changes in settings.py

    (see http://www.vanimpe.eu/2014/07/04/install-dionaeafr-web-frontend-dionaea-ubuntu/)

    1. Ion

      Hi Koen, thanks for sharing! I will update my blog post 🙂

      Regards,
      Ion

  8. Cristhoper

    Excuse my DionaeaFR not show statistics. What actions should I take?

    1. Ion

      Hello Christopher,
      unfortunately I can’t help you if you don’t provide any other details. Do you have any log lines or console output from DionaeaFR that you can paste here?

      Regards,
      Ion

  9. captin

    Hello, I’m having a problem accessing DionaeaFR through web browser. I receive “Operational error” error with this trace back :

    Environment:

    Request Method: GET

    Request URL: http://54.169.13.119:8000/

    Django Version: 1.7.1

    Python Version: 2.7.6

    Installed Applications:

    (‘django.contrib.auth’,

    ‘django.contrib.contenttypes’,

    ‘django.contrib.sessions’,

    ‘django.contrib.sites’,

    ‘django.contrib.messages’,

    ‘django.contrib.staticfiles’,

    ‘compressor’,

    ‘django_tables2’,

    ‘django_tables2_simplefilter’,

    ‘pagination’,

    ‘django.contrib.humanize’,

    ‘Web’)

    Installed Middleware:

    (‘django.middleware.gzip.GZipMiddleware’,

    ‘htmlmin.middleware.HtmlMinifyMiddleware’,

    ‘django.middleware.common.CommonMiddleware’,

    ‘django.contrib.sessions.middleware.SessionMiddleware’,

    ‘django.middleware.csrf.CsrfViewMiddleware’,

    ‘django.contrib.auth.middleware.AuthenticationMiddleware’,

    ‘django.contrib.messages.middleware.MessageMiddleware’,

    ‘django.middleware.clickjacking.XFrameOptionsMiddleware’,

    ‘pagination.middleware.PaginationMiddleware’)

    Traceback:

    File “/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py” in get_response

    98. resolver_match = resolver.resolve(request.path_info)

    File “/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py” in resolve

    343. for pattern in self.url_patterns:

    File “/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py” in url_patterns

    372. patterns = getattr(self.urlconf_module, “urlpatterns”, self.urlconf_module)

    File “/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py” in urlconf_module

    366. self._urlconf_module = import_module(self.urlconf_name)

    File “/usr/lib/python2.7/importlib/__init__.py” in import_module

    37. __import__(name)

    File “/opt/DionaeaFR/DionaeaFR/urls.py” in

    5. from Web.views.download import downloadIndex

    File “/opt/DionaeaFR/Web/views/download.py” in

    11. length = len(Download.objects.all())

    File “/usr/local/lib/python2.7/dist-packages/django/db/models/query.py” in __len__

    122. self._fetch_all()

    File “/usr/local/lib/python2.7/dist-packages/django/db/models/query.py” in _fetch_all

    966. self._result_cache = list(self.iterator())

    File “/usr/local/lib/python2.7/dist-packages/django/db/models/query.py” in iterator

    265. for row in compiler.results_iter():

    File “/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py” in results_iter

    700. for rows in self.execute_sql(MULTI):

    File “/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py” in execute_sql

    786. cursor.execute(sql, params)

    File “/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py” in execute

    81. return super(CursorDebugWrapper, self).execute(sql, params)

    File “/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py” in execute

    65. return self.cursor.execute(sql, params)

    File “/usr/local/lib/python2.7/dist-packages/django/db/utils.py” in __exit__

    94. six.reraise(dj_exc_type, dj_exc_value, traceback)

    File “/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py” in execute

    65. return self.cursor.execute(sql, params)

    File “/usr/local/lib/python2.7/dist-packages/django/db/backends/sqlite3/base.py” in execute

    485. return Database.Cursor.execute(self, query, params)

    Exception Type: OperationalError at /

    Exception Value: no such table: downloads

    Any idea what actions to be done to get tool up and running!!

    1. Ion

      Hi captin,
      well, as the error says it seems that the “downloads” table is missing from your database? Is this happening in HoneyDrive or your own installation btw?

      1. captin

        Yes I thought of it to be that sort of issue. But I tried my best to find any file creating the tables,I failed though. Well, I’m using my own installation of Dionaea with DionaeaFR for the visualization. if any extra info. needed just let me know & I’d post it here..Thanks

      2. captin

        Hi Ion,

        I’m still unable to solve the missing DB table. according to the tutorial it have no step for db tables creation. I don’t know if I have to create it my own, if yes what fields it has to have
        Please help !!!
        thanks

      3. captin

        Hi Ion,

        I’m still unable to solve the missing DB table. according to the tutorial it has no step for db tables creation. I don’t know if I have to create it my own, if yes what fields it has to have
        Please help !!!
        thanks

      4. Ion

        Hi captin. I don’t think it makes much sense to go back and forth here. I’m not sure where your problem lies. I think the best solution is to just download HoneyDrive, move your existing dionaea database to it (with a VirtualBox shared folder) to the correct folder (/opt/dionaea/var/dionaea/ if I am not mistaken) and run the installed DionaeaFR.

  10. Donny

    Could someone help me how to see full statistics. I am not able to view except connections and downloads. Help

  11. Alex

    Hi Ion,

    thanks for you post. I run into a error each time a try to do :
    root@vps:/opt/DionaeaFR# python manage.py collectstatic
    Traceback (most recent call last):
    File “manage.py”, line 12, in
    file(pidfile, ‘w’).write(pid)
    IOError: [Errno 2] No such file or directory: ‘/var/run/dionaeafr/dionaeafr.pid’

    Any idea what could be the problem ?

    thanks.

    1. Ryan

      my fix was sudo mv settings.py.dist settings.py

  12. Waseem

    Why there is 0 Malware Analized I have binaries in Malware samples but 0 malware in DionaeaFR graphs why its not displaying ????

  13. Munch

    Hello, I am also getting this erro, the same as Alex has gotten.

    File “manage.py”, line 12, in
    file(pidfile, ‘w’).write(pid)
    IOError: [Errno 2] No such file or directory: ‘/var/run/dionaeafr/dionaeafr.pid’

    Any idea what is causing this ?

    1. Munch

      Can anybody help with this

      1. Ion

        Hi, are you using HoneyDrive 3 or your own installation?

      2. Saint

        I am using my own installation and getting the same error

      3. Waseem

        Here is the solution that I have found and worked for me
        https://github.com/rubenespadas/DionaeaFR/issues/22

  14. Simon

    Hi, I am getting the same error as Munch and Alex. I am using my own installation from the guide above, not HoneyDrive, is there any solution?

  15. Ryan

    Traceback (most recent call last):

    File “manage.py”, line 7, in

    execute_from_command_line(sys.argv)

    File “/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py”, line 338, in execute_from_command_line

    utility.execute()

    File “/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py”, line 303, in execute

    settings.INSTALLED_APPS

    File “/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py”, line 48, in __getattr__

    self._setup(name)

    File “/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py”, line 44, in _setup

    self._wrapped = Settings(settings_module)

    File “/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py”, line 92, in __init__

    mod = importlib.import_module(self.SETTINGS_MODULE)

    File “/usr/lib/python2.7/importlib/__init__.py”, line 37, in import_module

    __import__(name)

    ImportError: No module named settings

  16. Ryan

    Nevermind! Got it!

  17. Saint

    I am seeing following error when I try to browse DionaeaFR
    database is locked

    /usr/local/lib/python2.7/dist-packages/django/db/backends/sqlite3/base.py in execute, line 451

    Can someone please help me with this error resolution

    1. morenike oniyide

      please have you been able to resolve this error, i am having the same problem

  18. morenike oniyide

    Please i am seeing the following error on honeydrive… Can anybody help with this please

    Request Method: GET

    Request URL: http://192.168.15.4:8000/

    Django Version: 1.6.5

    Exception Type: DatabaseError

    Exception Value:

    database disk image is malformed

    Exception Location: /usr/local/lib/python2.7/dist-packages/django/db/backends/sqlite3/base.py in execute, line 451

    Python Executable: /usr/bin/python

    Python Version: 2.7.3

    1. Ion

      Hi, I’m not sure why this is happening, I’ve never seen it before. If you google sqlite + the error message above you can find some potential solutions. Let me know if any of those worked for you. Best of luck.

  19. Paul Jany

    Hello,

    Followed this link and getting this error

    has no attribute ‘IPAddressField

    Downgrade Django version and downgraded to ‘1.4.2’

    Now, getting the error as,
    AttributeError: ‘module’ object has no attribute ‘python_2_unicode_compatible’

    Installed following pip packages. Still same issue…

    pip install -U six
    pip install -U git+https://github.com/sloria/TextBlob.git@dev

    1. Melissa Schmitz

      Changing IPAddressField to GenericIPAddressField seems to fix it for me (using the newest Django version)

  20. Alee Meerza

    Hi Ion,

    Nice blog and thanks for posting.

    I did everything as written and Dionaea is up and running but I am unable to run the server when I try to execute the runserver command I get (attributeerror ‘module’ object has no attribute ‘IPaddressfield’) this error I am unable to resolve it. Please help.

    Thanks,
    Q

  21. AJ

    Hi,

    When I try to start the server with python manage.py runserver 0.0.0.0:8000
    It is throwing the following error:

    from django.conf.urls import patterns, url
    ImportError: cannot import name patterns

    Any tip what could be the issue ?

    1. Ion

      Hi AJ, not sure what’s the problem here but this has some suggestions: https://stackoverflow.com/questions/22532743/cant-import-patterns-in-django – namely you might have installed some versions of Django and the modules that conflict with each other. If all else fail, just download HoneyDrive that has all of these preinstalled and preconfigured.

  22. ilham

    Hi,
    I had problem when run python manage.py 0.0.0.0:8000

    Unhandled exception in thread started by
    Traceback (most recent call last):
    File “/usr/local/lib/python2.7/dist-packages/django/utils/autoreload.py”, line 226, in wrapper
    fn(*args, **kwargs)
    File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/r unserver.py”, line 121, in inner_run
    self.check(display_num_errors=True)
    File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 374, in check
    include_deployment_checks=include_deployment_checks,
    File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 361, in _run_checks
    return checks.run_checks(**kwargs)
    File “/usr/local/lib/python2.7/dist-packages/django/core/checks/registry.py”, line 81, in run_checks
    new_errors = check(app_configs=app_configs)
    File “/usr/local/lib/python2.7/dist-packages/django/core/checks/urls.py”, line 14, in check_url_config
    return check_resolver(resolver)
    File “/usr/local/lib/python2.7/dist-packages/django/core/checks/urls.py”, line 24, in check_resolver
    for pattern in resolver.url_patterns:
    File “/usr/local/lib/python2.7/dist-packages/django/utils/functional.py”, line 35, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
    File “/usr/local/lib/python2.7/dist-packages/django/urls/resolvers.py”, line 3 13, in url_patterns
    patterns = getattr(self.urlconf_module, “urlpatterns”, self.urlconf_module)
    File “/usr/local/lib/python2.7/dist-packages/django/utils/functional.py”, line 35, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
    File “/usr/local/lib/python2.7/dist-packages/django/urls/resolvers.py”, line 3 06, in urlconf_module
    return import_module(self.urlconf_name)
    File “/usr/lib/python2.7/importlib/__init__.py”, line 37, in import_module
    __import__(name)
    File “/opt/DionaeaFR/DionaeaFR/urls.py”, line 1, in
    from django.conf.urls import patterns, url
    ImportError: cannot import name patterns

    Could someone help me?
    What should i do?

  23. tinictus

    Make sure Django version is 1.8

  1. HoneyDrive 0.2 Nectar edition released! » BruteForce Lab's Blog

    […] Installed DionaeaFR , a visualization tool which was recently presented in my previous post. […]

  2. Simplified reverse proxying using nginx «

    […] Visualizing Dionaea’s results with DionaeaFR (bruteforcelab.com) […]

  3. 【きょうのITレポ】ハニーポット可視化ツール、DionaeaFRを導入してみた | ショなんとかドットねっと

    […] そこで導入したのが、ハニーポット可視化ツールであるDionaeaFRとKippo-Graphです。 […]

  4. DionaeaFR: adding parameterized date range - BruteForce Lab's Blog

    […] in Python and uses the Django web framework. I have covered DionaeaFR in the past in my post Visualizing Dionaea’s results with DionaeaFR and of course I have included it in […]

  5. DionaeaFR を使ってDionaeaをグラフィカルにCheck it out

    […] BruteForce Lab’s Blog Visualizing Dionaea’s results with DionaeaFR Tahoo! […]

  6. DionaeaFR – A Window Into Your Honeypot – Execute Malware Blog

    […] first post is this one by Ion Koniaris who is the author of DionaeaFR. It’s mostly complete but, oddly, there are a […]

Leave a Reply

More in Honeypots, Malware, Visualization
dork.db for Glastopf web honeypot
Visualizing a cyber attack on a VOIP server
TekTip ep18 – HoneyDrive
HoneyDrive Desktop released!
NICT Daedalus Cyber-attack alert system
Close