Commit 360a2726 authored by cif2cif's avatar cif2cif

README

parent 58f6cc1d
# Minimal makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
SPHINXPROJ = soneti
SOURCEDIR = .
BUILDDIR = _build
# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
.PHONY: help Makefile
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
\ No newline at end of file
# Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
config: 014296aa2ea9b8791431c140fa457adc
tags: 645f666f9bcd5a90fca523b33c5a78b7
.. image:: img/header.png
:height: 6em
:target: http://demos.gsi.dit.upm.es/senpy
.. image:: https://travis-ci.org/gsi-upm/senpy.svg?branch=master
:target: https://travis-ci.org/gsi-upm/senpy
Senpy lets you create sentiment analysis web services easily, fast and using a well known API.
As a bonus, senpy services use semantic vocabularies (e.g. `NIF <http://persistence.uni-leipzig.org/nlp2rdf/>`_, `Marl <http://www.gsi.dit.upm.es/ontologies/marl>`_, `Onyx <http://www.gsi.dit.upm.es/ontologies/onyx>`_) and formats (turtle, JSON-LD, xml-rdf).
Have you ever wanted to turn your sentiment analysis algorithms into a service?
With senpy, now you can.
It provides all the tools so you just have to worry about improving your algorithms:
`See it in action. <http://demos.gsi.dit.upm.es/senpy>`_
Installation
------------
The stable version can be installed in three ways.
Through PIP
***********
.. code:: bash
pip install --user senpy
Alternatively, you can use the development version:
.. code:: bash
git clone git@github.com:gsi-upm/senpy
cd senpy
pip install --user .
If you want to install senpy globally, use sudo instead of the ``--user`` flag.
Docker Image
************
Build the image or use the pre-built one: ``docker run -ti -p 5000:5000 balkian/senpy --host 0.0.0.0 --default-plugins``.
To add custom plugins, add a volume and tell senpy where to find the plugins: ``docker run -ti -p 5000:5000 -v <PATH OF PLUGINS>:/plugins balkian/senpy --host 0.0.0.0 --default-plugins -f /plugins``
Usage
-----
However, the easiest and recommended way is to just use the command-line tool to load your plugins and launch the server.
.. code:: bash
senpy
or, alternatively:
.. code:: bash
python -m senpy
This will create a server with any modules found in the current path.
For more options, see the `--help` page.
Alternatively, you can use the modules included in senpy to build your own application.
Deploying on Heroku
-------------------
Use a free heroku instance to share your service with the world.
Just use the example Procfile in this repository, or build your own.
`DEMO on heroku <http://senpy.herokuapp.com>`_
For more information, check out the `documentation <http://senpy.readthedocs.org>`_.
------------------------------------------------------------------------------------
Acknowledgement
---------------
This development has been partially funded by the European Union through the MixedEmotions Project (project number H2020 655632), as part of the `RIA ICT 15 Big data and Open Data Innovation and take-up` programme.
.. image:: img/me.png
:target: http://mixedemotions-project.eu
:height: 100px
:alt: MixedEmotions Logo
.. image:: img/eu-flag.jpg
:height: 100px
:target: http://ec.europa.eu/research/participants/portal/desktop/en/opportunities/index.html
NIF API
=======
.. http:get:: /api
Basic endpoint for sentiment/emotion analysis.
**Example request**:
.. sourcecode:: http
GET /api?input=I%20love%20GSI HTTP/1.1
Host: localhost
Accept: application/json, text/javascript
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"@context": [
"http://127.0.0.1/static/context.jsonld",
],
"analysis": [
{
"@id": "SentimentAnalysisExample",
"@type": "marl:SentimentAnalysis",
"dc:language": "en",
"marl:maxPolarityValue": 10.0,
"marl:minPolarityValue": 0.0
}
],
"domain": "wndomains:electronics",
"entries": [
{
"opinions": [
{
"prov:generatedBy": "SentimentAnalysisExample",
"marl:polarityValue": 7.8,
"marl:hasPolarity": "marl:Positive",
"marl:describesObject": "http://www.gsi.dit.upm.es",
}
],
"nif:isString": "I love GSI",
"strings": [
{
"nif:anchorOf": "GSI",
"nif:taIdentRef": "http://www.gsi.dit.upm.es"
}
]
}
]
}
:query i input: No default. Depends on informat and intype
:query f informat: one of `turtle` (default), `text`, `json-ld`
:query t intype: one of `direct` (default), `url`
:query o outformat: one of `turtle` (default), `text`, `json-ld`
:query p prefix: prefix for the URIs
:query algo algorithm: algorithm/plugin to use for the analysis. For a list of options, see :http:get:`/api/plugins`. If not provided, the default plugin will be used (:http:get:`/api/plugins/default`).
:reqheader Accept: the response content type depends on
:mailheader:`Accept` header
:resheader Content-Type: this depends on :mailheader:`Accept`
header of request
:statuscode 200: no error
:statuscode 404: service not found
.. http:post:: /api
The same as :http:get:`/api`.
.. http:get:: /api/plugins
Returns a list of installed plugins.
**Example request**:
.. sourcecode:: http
GET /api/plugins HTTP/1.1
Host: localhost
Accept: application/json, text/javascript
**Example response**:
.. sourcecode:: http
{
"@context": {
...
},
"sentiment140": {
"name": "sentiment140",
"is_activated": true,
"version": "0.1",
"extra_params": {
"@id": "extra_params_sentiment140_0.1",
"language": {
"required": false,
"@id": "lang_sentiment140",
"options": [
"es",
"en",
"auto"
],
"aliases": [
"language",
"l"
]
}
},
"@id": "sentiment140_0.1"
},
"rand": {
"name": "rand",
"is_activated": true,
"version": "0.1",
"extra_params": {
"@id": "extra_params_rand_0.1",
"language": {
"required": false,
"@id": "lang_rand",
"options": [
"es",
"en",
"auto"
],
"aliases": [
"language",
"l"
]
}
},
"@id": "rand_0.1"
}
}
.. http:get:: /api/plugins/<pluginname>
Returns the information of a specific plugin.
**Example request**:
.. sourcecode:: http
GET /api/plugins/rand HTTP/1.1
Host: localhost
Accept: application/json, text/javascript
**Example response**:
.. sourcecode:: http
{
"@id": "rand_0.1",
"extra_params": {
"@id": "extra_params_rand_0.1",
"language": {
"@id": "lang_rand",
"aliases": [
"language",
"l"
],
"options": [
"es",
"en",
"auto"
],
"required": false
}
},
"is_activated": true,
"name": "rand",
"version": "0.1"
}
.. http:get:: /api/plugins/default
Return the information about the default plugin.
.. http:get:: /api/plugins/<pluginname>/{de}activate
{De}activate a plugin.
**Example request**:
.. sourcecode:: http
GET /api/plugins/rand/deactivate HTTP/1.1
Host: localhost
Accept: application/json, text/javascript
**Example response**:
.. sourcecode:: http
{
"@context": {},
"message": "Ok"
}
.. Senpy documentation master file, created by
sphinx-quickstart on Tue Feb 24 08:57:32 2015.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to Senpy's documentation!
=================================
Contents:
.. toctree::
installation
usage
api
plugins
:maxdepth: 2
Installation
------------
The stable version can be installed in three ways.
Through PIP
***********
.. code:: bash
pip install --user senpy
Alternatively, you can use the development version:
.. code:: bash
git clone git@github.com:gsi-upm/senpy
cd senpy
pip install --user .
If you want to install senpy globally, use sudo instead of the ``--user`` flag.
Docker Image
************
Build the image or use the pre-built one: ``docker run -ti -p 5000:5000 balkian/senpy --host 0.0.0.0 --default-plugins``.
To add custom plugins, add a volume and tell senpy where to find the plugins: ``docker run -ti -p 5000:5000 -v <PATH OF PLUGINS>:/plugins balkian/senpy --host 0.0.0.0 --default-plugins -f /plugins``
Developing new plugins
----------------------
Plugins Interface
=================
The basic methods in a plugin are:
* __init__
* activate: used to load memory-hungry resources
* deactivate: used to free up resources
* analyse: called in every user requests. It takes in the parameters supplied by a user and should return a senpy Response.
Plugins are loaded asynchronously, so don't worry if the activate method takes too long. The plugin will be marked as activated once it is finished executing the method.
F.A.Q.
======
If I'm using a classifier, where should I train it?
???????????????????????????????????????????????????
Training a classifier can be time time consuming. To avoid running the training unnecessarily, you can use ShelfMixin to store the classifier. For instance:
.. code:: python
from senpy.plugins import ShelfMixin, SenpyPlugin
class MyPlugin(ShelfMixin, SenpyPlugin):
def train(self):
''' Code to train the classifier
'''
# Here goes the code
# ...
return classifier
def activate(self):
if 'classifier' not in self.sh:
classifier = self.train()
self.sh['classifier'] = classifier
self.classifier = self.sh['classifier']
def deactivate(self):
self.close()
You can speficy a 'shelf_file' in your .senpy file. By default the ShelfMixin creates a file based on the plugin name and stores it in that plugin's folder.
Where can I find more code examples?
????????????????????????????????????
See: `<http://github.com/gsi-upm/senpy-plugins-community>`_.
Usage
-----
The easiest and recommended way is to just use the command-line tool to load your plugins and launch the server.
.. code:: bash
senpy
Or, alternatively:
.. code:: bash
python -m senpy
This will create a server with any modules found in the current path.
For more options, see the `--help` page.
Alternatively, you can use the modules included in senpy to build your own application.
.. soneti documentation master file, created by
sphinx-quickstart on Wed Jul 12 11:08:03 2017.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to soneti's documentation!
==================================
.. toctree::
:maxdepth: 2
:caption: Contents:
what-is-soneti
uses-soneti
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
.. Undocumented documentation master file, created by
sphinx-quickstart on Mon Apr 24 16:53:33 2017.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to Undocumented's documentation!
========================================
This demo shows a picture.
Its only purpose is to show you how to document a project from scratch.
.. toctree::
:maxdepth: 2
:caption: Contents:
installation
usage
Indices and tables
==================
* :ref:`search`
Installing undocumented
=======================
To get the source code for this demo repository just `download the file <https://lab.cluster.gsi.dit.upm.es/docs/readthedocs/repository/archive.zip?ref=master>`_ or clone the repository with git:
.. code-block:: bash
git clone https://lab.cluster.gsi.dit.upm.es/docs/readthedocs/tree/master
Using undocumented
==================
Open your browser with ``firefox docs/_build/html/index.html`` and enjoy the beauty of our image:
.. image:: ../iamundocumented.jpg
==============
Uses of Soneti
==============
We have used Soneti in a number of projects, including companies and research centers.
Here we include some videos to provide a better understanding of its capabilities. You can consult an updated list at GSI_ YouTube channel
.. _GSI: http://www.gsi.dit.upm.es/index.php/es/tecnologias/canal-gsi
Crawling
--------
This video shows you the basic behaviour of **GSICrawler**. GSICrawler provides an interface to crawl a number of web sites, based on Scrapy_. The results of Scrapy are returned as a JSON-LD (JSONLD_, JSON for Linking Data). This file can be processed and stored later by the Analysis modules. In this way, the scrapped information is mapped onto a number of ontologies and can be linked and queried. If you need to scrape a new web site, a new spider should be developed in Python.
.. _Scrapy: https://scrapy.org/
.. _JSONLD: https://json-ld.org/
.. raw:: html
<div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; height: auto;">
<iframe width="560" height="315" src="https://www.youtube.com/embed/MNg5YKLI2uc" frameborder="0" allowfullscreen></iframe>
<div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; height: auto;">
<iframe width="560" height="315" src="https://www.youtube.com/embed/OQZbcQFeVIQ" frameborder="0" allowfullscreen></iframe>
</div>
Sentiment analysis
------------------
The tool **Senpy** has been used for sentiment and emotion analysis from social networks. **Senpy** provides semantic interoperability across different analyzers, so you do not need to change your code to use a new analyzer. The main advantages of Senpy (Senpy_) are that (i) it provides out-of-the-box facilities for transforming the analysis into linked data and (ii) it follows a plugin-in architecture which makes it easy to extend.
.. _Senpy: http://senpy.readthedocs.io
Here we show some examples of sentiment and emotion analysis of Tweets in the football domain and political web pages in Facebook.
.. raw:: html
<div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; height: auto;">
<iframe width="560" height="315" src="https://www.youtube.com/embed/EvIJyEd8wyE" frameborder="0" allowfullscreen></iframe>
</div>
<div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; height: auto;">
<iframe width="560" height="315" src="https://www.youtube.com/embed/XSmGCQOgrn8" frameborder="0" allowfullscreen></iframe>
</div>
User profiling
-------------------
The tool **Senpy** has also been used for user profiling, including personality, gender or age, as shown below.
.. raw:: html
<div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; height: auto;">
<iframe width="560" height="315" src="https://www.youtube.com/embed/v0tTa5J1dFc" frameborder="0" allowfullscreen></iframe>
</div>
<div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; height: auto;">
<iframe width="560" height="315" src="https://www.youtube.com/embed/A5tFS4IvACA" frameborder="0" allowfullscreen></iframe>
</div>
Visualization and Semantic query
--------------------------------
Visualization is done with the tool **Sefarad** which is based on W3C Web Components. **Sefarad** (Sefarad_) provides a simple way to create dashboards with textual and faceted search.
.. _Sefarad:
Here we show some examples of dashboards that show sentiment and emotion analysis for financial news and tourism.
.. raw:: html
<div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; height: auto;">