senpy issueshttps://lab.gsi.upm.es/senpy/senpy/-/issues2023-09-22T22:25:53Zhttps://lab.gsi.upm.es/senpy/senpy/-/issues/12Conversion plugins in playground2023-09-22T22:25:53ZJ. Fernando Sánchezjf.sanchez@upm.esConversion plugins in playgroundWe should filter them out, because they can't be called on their own.We should filter them out, because they can't be called on their own.J. Fernando Sánchezjf.sanchez@upm.esJ. Fernando Sánchezjf.sanchez@upm.eshttps://lab.gsi.upm.es/senpy/senpy/-/issues/4Extra_params is duplicated in some plugins2023-09-22T22:25:53ZJ. Fernando Sánchezjf.sanchez@upm.esExtra_params is duplicated in some plugins*Created by: balkian*
*Created by: balkian*
https://lab.gsi.upm.es/senpy/senpy/-/issues/5Improve threading2023-09-22T22:25:53ZJ. Fernando Sánchezjf.sanchez@upm.esImprove threading*Created by: balkian*
We are using gevent and some plugins are blocking on I/O without monkey patching. That means some operations are actually blocking the whole service.
*Created by: balkian*
We are using gevent and some plugins are blocking on I/O without monkey patching. That means some operations are actually blocking the whole service.
https://lab.gsi.upm.es/senpy/senpy/-/issues/3Deadlock in ShelfPlugin2023-09-22T22:25:53ZJ. Fernando Sánchezjf.sanchez@upm.esDeadlock in ShelfPlugin*Created by: balkian*
When a shelf plugin is not properly closed, reusing the shelf file results in an error:
```
DBPageNotFoundError: (-30986, 'BDB0075 DB_PAGE_NOTFOUND: Requested page not found')
```
*Created by: balkian*
When a shelf plugin is not properly closed, reusing the shelf file results in an error:
```
DBPageNotFoundError: (-30986, 'BDB0075 DB_PAGE_NOTFOUND: Requested page not found')
```
https://lab.gsi.upm.es/senpy/senpy/-/issues/52Remove parameter calculation from the UI2019-04-11T10:07:32ZJ. Fernando Sánchezjf.sanchez@upm.esRemove parameter calculation from the UIOnce senpy/senpy#51 is implemented, the UI should ask the server what parameters to use with each plugin (or combination of plugins).
Optionally, the results should be cached in the client to avoid unnecessary calls.Once senpy/senpy#51 is implemented, the UI should ask the server what parameters to use with each plugin (or combination of plugins).
Optionally, the results should be cached in the client to avoid unnecessary calls.Senpy 1.0.0J. Fernando Sánchezjf.sanchez@upm.esJ. Fernando Sánchezjf.sanchez@upm.eshttps://lab.gsi.upm.es/senpy/senpy/-/issues/51Calculate and expose list of parameters for the combination of plugins2018-12-07T17:33:44ZJ. Fernando Sánchezjf.sanchez@upm.esCalculate and expose list of parameters for the combination of pluginsFor a long time now, it is also possible to chain several plugins.
The `/plugins` API returns the parameters for each plugin, but there is no easy way to know what parameters can be sent to the chain.
There may be some discrepancies bet...For a long time now, it is also possible to chain several plugins.
The `/plugins` API returns the parameters for each plugin, but there is no easy way to know what parameters can be sent to the chain.
There may be some discrepancies between the values that each plugin accepts.
I propose the following:
- Accept the union of both sets of parameters
- For shared parameters, provide the intersection of valid options
- Allow setting different parameters for each plugin via special names. e.g. for plugin `sentiment`, one could set its `language` parameter with `sentiment-language`.
The parameters should be listed with the API as well.
#50 should probably be implemented first.
Note that parameter validation/handling on plugins execution should also change, but I think it will simplify things for the client and the UI.Senpy 1.0.0J. Fernando Sánchezjf.sanchez@upm.esJ. Fernando Sánchezjf.sanchez@upm.eshttps://lab.gsi.upm.es/senpy/senpy/-/issues/57Refactor conversion2018-11-22T16:50:24ZJ. Fernando Sánchezjf.sanchez@upm.esRefactor conversionThe conversion mechanism was developed with emotion conversion in mind.
In order to enable other types of conversion, it needs to be refactored or extended.
My first thought was to mimic what we are doing with the analysis, and have an ...The conversion mechanism was developed with emotion conversion in mind.
In order to enable other types of conversion, it needs to be refactored or extended.
My first thought was to mimic what we are doing with the analysis, and have an analogous method such as `convert_entry`.
The problem with this approach is that whereas an analysis is always run, a conversion may not always need to be.
As a solution, we could add an intermediate step that selects conversions that need to be applied, and run them in series.
The selection could be handled somewhere in the core (e.g. in the extensions module) or delegated to each conversion plugin, with something like a `should_convert(entry, params, other_plugins...)` method.
The `other_plugins` part could tell the plugin what other conversions are already going to be applied.
In summary, these are the main ideas:
- [ ] Avoid unnecessary conversions. Until now, we've been using only one plugin to convert each EmotionSet. i.e. if there are two candidates to convert an EmotionSet, only the first one is run.
- [ ] Keep it general enough to allow for any type of conversion (sentiment, emotion...), including conversions that act as "filters" (getting the emotion with the highest value).
- [ ] Allow users to specify the conversion they want (explicit is better than implicit)
- [ ] Avoid unnecessary computations that might slow down the serviceJ. Fernando Sánchezjf.sanchez@upm.esJ. Fernando Sánchezjf.sanchez@upm.eshttps://lab.gsi.upm.es/senpy/senpy/-/issues/56Exception when using POST2018-11-07T15:14:04ZJ. Fernando Sánchezjf.sanchez@upm.esException when using POSTExample:
```
ERROR:flask.app:Exception on /api/ [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/usr/loca...Example:
```
ERROR:flask.app:Exception on /api/ [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.5/site-packages/flask/_compat.py", line 35, in reraise
raise value
File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/src/app/senpy/blueprints.py", line 177, in decorated_function
prefix=params.get('prefix', encoded_url()),
File "/usr/src/app/senpy/blueprints.py", line 72, in encoded_url
hash(frozenset(request.form.params().items()))
AttributeError: 'ImmutableMultiDict' object has no attribute 'params'
```J. Fernando Sánchezjf.sanchez@upm.esJ. Fernando Sánchezjf.sanchez@upm.eshttps://lab.gsi.upm.es/senpy/senpy/-/issues/50Make the algorithm part of the URL2018-08-17T09:24:21ZJ. Fernando Sánchezjf.sanchez@upm.esMake the algorithm part of the URLAdd `/api/<plugin>` as an alternative to `/api?algo=<plugin>`.
For pipelines, we could do something like `/api/<plugin1>/<plugin2>/...` or `/api/<plugin1>+<plugin2>+...`Add `/api/<plugin>` as an alternative to `/api?algo=<plugin>`.
For pipelines, we could do something like `/api/<plugin1>/<plugin2>/...` or `/api/<plugin1>+<plugin2>+...`Senpy 1.0.0J. Fernando Sánchezjf.sanchez@upm.esJ. Fernando Sánchezjf.sanchez@upm.eshttps://lab.gsi.upm.es/senpy/senpy/-/issues/44Add basic evaluation with GSITK2018-04-25T08:18:01ZJ. Fernando Sánchezjf.sanchez@upm.esAdd basic evaluation with GSITKIgnacio CorcueraIgnacio Corcuerahttps://lab.gsi.upm.es/senpy/senpy/-/issues/1Load plugin requirements automatically2018-03-23T11:48:07ZJ. Fernando Sánchezjf.sanchez@upm.esLoad plugin requirements automatically*Created by: balkian*
Right now, if a plugin has extra requirements that are not installed, it fails to load.
*Created by: balkian*
Right now, if a plugin has extra requirements that are not installed, it fails to load.
https://lab.gsi.upm.es/senpy/senpy/-/issues/45Option arguments are not passed to the API2018-01-08T18:09:27ZJ. Fernando Sánchezjf.sanchez@upm.esOption arguments are not passed to the APITry with sentiment-meaningcloud. The language parameter is recognized as a text input instead of a select.Try with sentiment-meaningcloud. The language parameter is recognized as a text input instead of a select.Senpy 1.0.0J. Fernando Sánchezjf.sanchez@upm.esJ. Fernando Sánchezjf.sanchez@upm.eshttps://lab.gsi.upm.es/senpy/senpy/-/issues/23Add section about client use2018-01-08T18:04:44ZJ. Fernando Sánchezjf.sanchez@upm.esAdd section about client useAdd a short section with instructions on client use, preferably as an ipython notebook.
* [x] Add nbsphinx
* [x] Add a quick example using the demo endpoint (senpy.cluster)
* [x] Change the endpoint to a local oneAdd a short section with instructions on client use, preferably as an ipython notebook.
* [x] Add nbsphinx
* [x] Add a quick example using the demo endpoint (senpy.cluster)
* [x] Change the endpoint to a local oneSenpy 1.0.0Manuel García-AmadoManuel García-Amado2017-04-07https://lab.gsi.upm.es/senpy/senpy/-/issues/47Wrong version of pip2017-12-13T14:33:02ZJ. Fernando Sánchezjf.sanchez@upm.esWrong version of pipIn systems with python2 and python3, senpy uses the `pip` executable regardless of what version of python is running.
As a consequence, the requirements are only installed for the default version of python.In systems with python2 and python3, senpy uses the `pip` executable regardless of what version of python is running.
As a consequence, the requirements are only installed for the default version of python.Senpy 1.0.0J. Fernando Sánchezjf.sanchez@upm.esJ. Fernando Sánchezjf.sanchez@upm.eshttps://lab.gsi.upm.es/senpy/senpy/-/issues/46Make data folder configurable2017-12-13T14:33:02ZJ. Fernando Sánchezjf.sanchez@upm.esMake data folder configurableInstead of hardcoding paths in the senpy file, allow the use of a DATA_FOLDER / SENPY_DATA variable, like in shelf plugins.
There are two (non exclusive) alternatives to integrate this feature in the plugin definition:
* Prepend SENPY_...Instead of hardcoding paths in the senpy file, allow the use of a DATA_FOLDER / SENPY_DATA variable, like in shelf plugins.
There are two (non exclusive) alternatives to integrate this feature in the plugin definition:
* Prepend SENPY_DATA to relative paths
* Use the variable name in the path `$SENPY_DATA/blahblahblah`.
The second one is explicit, whereas the first one makes changing paths relative to the CWD to SENPY_DATA seamless.Senpy 1.0.0J. Fernando Sánchezjf.sanchez@upm.esJ. Fernando Sánchezjf.sanchez@upm.eshttps://lab.gsi.upm.es/senpy/senpy/-/issues/48Error in split plugin2017-12-13T14:33:02ZManuel García-AmadoError in split pluginThere is an error in split plugin concerning the span tokenize function from the nltk LineTokenizerThere is an error in split plugin concerning the span tokenize function from the nltk LineTokenizerManuel García-AmadoManuel García-Amadohttps://lab.gsi.upm.es/senpy/senpy/-/issues/49Missing extra params when using playground2017-12-13T14:05:52ZManuel García-AmadoMissing extra params when using playgroundSome parameters are not passed to senpy when using the playground.
![Captura_de_pantalla_2017-12-12_a_las_15.26.46](/uploads/ae54eaaed6276b40425a69baf1cded4f/Captura_de_pantalla_2017-12-12_a_las_15.26.46.png)![Captura_de_pantalla_2017-1...Some parameters are not passed to senpy when using the playground.
![Captura_de_pantalla_2017-12-12_a_las_15.26.46](/uploads/ae54eaaed6276b40425a69baf1cded4f/Captura_de_pantalla_2017-12-12_a_las_15.26.46.png)![Captura_de_pantalla_2017-12-12_a_las_15.26.58](/uploads/d18906c07039ad695f66d5b43b92e2d3/Captura_de_pantalla_2017-12-12_a_las_15.26.58.png)https://lab.gsi.upm.es/senpy/senpy/-/issues/36Estimate VAD by weighted average2017-09-27T14:19:31ZJ. Fernando Sánchezjf.sanchez@upm.esEstimate VAD by weighted averageSee https://github.com/gsi-upm/senpy/issues/31See https://github.com/gsi-upm/senpy/issues/31Senpy 1.0.0J. Fernando Sánchezjf.sanchez@upm.esJ. Fernando Sánchezjf.sanchez@upm.eshttps://lab.gsi.upm.es/senpy/senpy/-/issues/33Using both entry.text and entry.nif__isString2017-08-23T16:28:14ZJ. Fernando Sánchezjf.sanchez@upm.esUsing both entry.text and entry.nif__isStringSome parts of the docs refer to "entry.text", some to "entry.nif__isString". We need plugins to be consistent.
There are several options:
## Use `nif:isString`
We would have to:
- Check code
- Check docs
## Use `text` everywhere, add ...Some parts of the docs refer to "entry.text", some to "entry.nif__isString". We need plugins to be consistent.
There are several options:
## Use `nif:isString`
We would have to:
- Check code
- Check docs
## Use `text` everywhere, add it to the context and adapt all code and docs
Tasks:
- Check code
- Check docs
- Add `text` to `context.jsonld`
## Create mappings/aliases
Either as properties (more efficient) or as a dictionary of aliases that can be checked in `__get/setitem__` or similar. There are several options for this.
- Ad-hoc properties in the entry model
- Add aliases to the schema
- Load aliases by reading the contextSenpy 1.0.0J. Fernando Sánchezjf.sanchez@upm.esJ. Fernando Sánchezjf.sanchez@upm.eshttps://lab.gsi.upm.es/senpy/senpy/-/issues/37Add a basic chunk plugin2017-08-23T16:24:19ZJ. Fernando Sánchezjf.sanchez@upm.esAdd a basic chunk pluginIt'd be useful to have a plugin that splits a long text into a series of smaller ones (e.g. sentences, paragraphs...).
Ideally, it would take a `delimiter` or similar argument.
It should be trivial to implement this plugin.
The only com...It'd be useful to have a plugin that splits a long text into a series of smaller ones (e.g. sentences, paragraphs...).
Ideally, it would take a `delimiter` or similar argument.
It should be trivial to implement this plugin.
The only complication would be handling the `id` of the pieces, since they should follow one of NIF schemes.
Once implemented, it could be used like this: `http://localhost:5000/api?i=hello there&algorithm=split,rand`.Senpy 1.0.0Manuel García-AmadoManuel García-Amado