Skip to content

GitLab

Commit cba5d026 authored by J. Fernando Sánchez's avatar J. Fernando Sánchez
Browse files

Added pipeline and demo

parent 97cead6b
Show whitespace changes
Inline Side-by-side
.gitignore
View file @ cba5d026
*.html~
.*
*~
bower_components
.gitmodules 0 → 100644
View file @ cba5d026
[submodule "elements/number-chart"]
path = elements/number-chart
url = ssh://git@lab.cluster.gsi.dit.upm.es:2200/sefarad/number-chart.git
Dockerfile 0 → 100644
View file @ cba5d026
from node:7.10.0
ENV NODE_PATH=/tmp/node_modules APP_NAME=dashboard-tourpedia
# Install dependencies first to use cache
RUN npm install -g http-server bower
ADD bower.json /usr/src/bower.json
RUN cd /usr/src && \
bower install --allow-root
ADD . /usr/src/app
WORKDIR /usr/src/app/
CMD ["/usr/src/app/init.sh"]
README.md 0 → 100644
View file @ cba5d026
`Sefarad dashboard for the tourpedia data.
## Usage
This web component accepts the following parameters:
```html
<dashboard-tourpedia
client="<!-- elasticsearch client -->"
</dashboard-tourpedia>
```
See the `dashboard-tourpedia.html` and `demo/index.html` for more information.
## Installation
This web component is available in bower.
```bash
$ bower install dashboard-tourpedia
```
This command will install it inside `bower_components` folder
## Development
Requirements:
* Docker
* Docker-compose
The docker-compose file can be used to test the component and to develop it.
Simply run:
```
docker compose up
```
And go to http://127.0.0.1:8080/demo/index.html
The docker-compose file mounts the current directory in the docker container, so every change you make to files locally will be reflected immediately in the browser.
If you add new dependencies to the compponent (through the `bower.json` file), you need to either run `bower install` within the container or recreate the image, like so:
```
docker compose up --build
```
Or:
```
docker exec web bower install
```
Note that the component assumes all dependencies are added in `../`.
This is the structure the component will find when installed as a dependency with bower.
To mimic that structure, the `init.sh` script automatically links the bower package.
bower.json
View file @ cba5d026
......@@ -14,14 +14,22 @@
"material-search": "material-search#*",
"leaflet-maps": "leaflet-maps#*",
"paper-tabs": "paper-tabs#*",
"yasgui-polymer": "yasgui-polymer#*",
"reviews-table": "reviews-table#*",
"number-chart": "number-chart#*",
"number-chart": "number-chart#^1.1.1",
"google-apis": "GoogleWebComponents/google-apis#^1.0.0",
"google-chart-elasticsearch": "google-chart-elasticsearch#^1.1.0",
"iron-icons": "PolymerElements/iron-icons#^1.0.0",
"paper-icon-button": "PolymerElements/paper-icon-button#^1.0.0",
"polymer": "Polymer/polymer#^1.1.0"
"polymer": "Polymer/polymer#^1.1.0",
"jquery": "jquery#^2.1.4",
"elasticsearchjs-import": "DigElements/elasticsearchjs-import#~1.0.0",
"elasticjs-import": "DigElements/elasticjs-import#~1.0.1",
"webcomponentsjs": "< 1.0.0"
},
"license": "MIT",
"resolutions": {
"webcomponentsjs": "0.7.24"
},
"license": "Apache-2.0",
"homepage": "https://lab.cluster.gsi.dit.upm.es/sefarad/dashboard-tourpedia"
}
dashboard-tourpedia.html
View file @ cba5d026
<link rel="import" href="../../bower_components/polymer/polymer.html">
<link rel="import" href="../../bower_components/material-search/material-search.html">
<link rel="import" href="../../bower_components/paper-material/paper-material.html">
<link rel="import" href="../../bower_components/google-chart-elk/google-chart.html">
<link rel="import" href="../../bower_components/paper-tabs/paper-tabs.html">
<link rel="import" href="../../bower_components/yasgui/yasgui.html">
<link rel="import" href="../../bower_components/reviews-table/reviews-table.html">
<link rel="import" href="../../bower_components/number-chart/number-chart.html">
<link rel="import" href="../../bower_components/leaflet-map/leaflet-map.html">
<link rel="import" href="../polymer/polymer.html">
<link rel="import" href="../material-search/material-search.html">
<link rel="import" href="../paper-material/paper-material.html">
<link rel="import" href="../google-chart-elasticsearch/google-chart.html">
<link rel="import" href="../paper-tabs/paper-tabs.html">
<link rel="import" href="../reviews-table/reviews-table.html">
<link rel="import" href="../number-chart/number-chart.html">
<dom-module id="dashboard-tourpedia">
......@@ -76,25 +75,42 @@
</number-chart>
</div>
</div>
<!-- <div style="width: 100%; display: inline-block"> -->
<!-- <div style="width: 23%; margin-right: 2.6%; float: left"> -->
<!-- <number-chart -->
<!-- data="{{data}}" -->
<!-- object="londres" -->
<!-- title="Londres" -->
<!-- icon="maps:local-dining" -->
<!-- aggkey="location" -->
<!-- stylebg="bg-yellow"> -->
<!-- </number-chart> -->
<!-- </div> -->
<!-- <div style="width: 23%; margin-right: 2.6%; float: left"> -->
<!-- <number-chart -->
<!-- data="{{data}}" -->
<!-- object="madrid" -->
<!-- aggkey="location" -->
<!-- title="Madrid" -->
<!-- icon="social:whatshot" -->
<!-- stylebg="bg-green"> -->
<!-- </number-chart> -->
<!-- </div> -->
<!-- <div style="width: 23%; float: left"> -->
<!-- <number-chart -->
<!-- icon="maps:hotel" -->
<!-- data="{{data}}" -->
<!-- title="Paris" -->
<!-- object="paris" -->
<!-- aggkey="location" -->
<!-- stylebg="bg-aqua"> -->
<!-- </number-chart> -->
<!-- </div> -->
<!-- </div> -->
<div style="width: 100%; display: inline-block">
<div class="chart_container" style="left: 0; float: left">
<google-chart
field="category"
datos="{{data}}"
id='pie-chart3'
extra-id='pie-chart3'
type='pie'
filters="{{filters}}"
icon='room'
param="{{param}}"
options='{"title": "Different types of places"}'
cols='[{"label": "Sentiment", "type": "string"},{"label": "Count", "type": "number"}]'
rows='[ ["Jan", 31],["Feb", 28],["Mar", 31],["Apr", 30],["May", 31],["Jun", 30] ]'>
</google-chart>
</div>
<div class="chart_container" style="right: 0; float: right">
<google-chart
datos="{{data}}"
data="{{data}}"
field="location"
id='pie-chart4'
extra-id='pie-chart4'
......@@ -104,77 +120,29 @@
param='{{param}}'
options='{"title": "Places at different cities"}'
cols='[{"label": "Place", "type": "string"},{"label": "Count", "type": "number"}]'
rows='[ ["Jan", 31],["Feb", 28],["Mar", 31],["Apr", 30],["May", 31],["Jun", 30] ]'>
</google-chart>
</div>
</div>
<div style="width: 100%">
<div class="chart_container" style="left: 0; float: left">
<google-chart
datos="{{data}}"
field="numReviews"
id='column-chart1'
extra-id='column-chart1'
type='column'
filters="{{filters}}"
icon='maps:rate-review'
param='{{param}}'
optionsbi='{"legend": "none"}'
options='{"title": "Count of number of reviews"}'
cols='[{"label": "Reviews", "type": "number"},{"label": "Count", "type": "number"}]'
rows='[ [0, 31],[1, 28],[2, 31],[3, 30],[4, 30],[5, 31],[6, 30] ]'>
</google-chart>
</div>
<div class="chart_container" style="right: 0; float: right">
<google-chart
datos="{{data}}"
field="polarity"
fields='["category", "location"]'
id='bar-chart3'
extra-id='bar-chart3'
type='bar'
field="category"
data="{{data}}"
id='pie-chart3'
extra-id='pie-chart3'
type='pie'
filters="{{filters}}"
icon='maps:local-play'
param='{{param}}'
optionsbi='{"legend": "none"}'
options='{"title": "Count of different polarities"}'
cols='[{"label": "Polarity", "type": "number"},{"label": "Count", "type": "number"}]'
rows='[ ["Jan", 31],["Feb", 28],["Mar", 31],["Apr", 30],["May", 31],["Jun", 30] ]'>
icon='room'
param="{{param}}"
options='{"title": "Different types of places"}'
cols='[{"label": "Sentiment", "type": "string"},{"label": "Count", "type": "number"}]'
</google-chart>
</div>
<div style="clear: both"></div>
</div>
<div style="width: 100%">
<div class="chart_container" style="left: 0; float: left">
<reviews-table
data="{{ids}}"
icon='social:people'>
</reviews-table>
</div>
<div class="chart_container" style="right: 0; float: right">
<leaflet-map fit-to-markers>
<template is="dom-repeat" items="[[getPoints(places.*)]]" as="place">
<leaflet-marker latitude="{{place.lat}}" longitude="{{place.lng}}" title="{{place.name}}">
<b>Name:</b>
<span>{{place.name}}</span>
</leaflet-marker>
</template>
</leaflet-map>
</div>
<div style="clear: both"></div>
</div>
</div>
<div>
<yasgui-ui
endpoint="http://fuseki-tourpedia.cluster.gsi.dit.upm.es/tourpedia/query"
queries="{{queries}}">
</yasgui-ui>
</div>
</iron-pages>
</template>
<script>
var filtered = false;
var ready = false;
Polymer({
is: 'dashboard-tourpedia',
properties: {
......@@ -188,16 +156,14 @@
/*value: ["481","482","32512","32420","91963","96677","160677","131425","322429","322612"]*/
},
data:{
type: Object
},
query: {
type: String,
observer: '_queryChanged'
client: {
type: Object,
notify: true,
observer: '_clientChanged'
},
fields: {
type: Array,
value: function() { return []; }
......@@ -221,7 +187,7 @@
],
ready: function(){
this.queryDefault();
console.log("ready");
},
addPlace: function(event) {
......@@ -237,143 +203,20 @@
return f.base;
},
_queryChanged: function() {
console.log("_queryChanged")
this.query ? this.queryChange(this.query) : this.queryDefault();
},
queryChange: function(value) {
console.log("_filtersChange")
var that = this;
var client = new $.es.Client({
hosts: this.host
});
client.search({
// undocumented params are appended to the query string
index: "tourpedia",
type: "places",
body: {
size: 10,
query: {
multi_match: {
fields: ["category","location"],
query: this.query
}
},
aggs: {
category: {
terms: {
field: "category",
order: {
_count: "desc"
}
}
},
location: {
terms: {
field: "location",
order: {
_count: "desc"
}
}
},
numReviews: {
terms: {
field: "numReviews",
order: {
_count: "desc"
}
}
},
polarity: {
terms: {
field: "polarity",
order: {
_count: "desc"
}
}
}
}
}
}).then(function (resp) {
var myids = []
resp.hits.hits.forEach(function(entry){myids.push(entry._id)})
that.ids = myids;
that.data = resp;
});
},
queryDefault: function() {
var that = this;
var client = new $.es.Client({
hosts: this.host
});
client.search({
// undocumented params are appended to the query string
index: "tourpedia",
type: "places",
body: {
size: 10,
query: {
match_all: {}
_clientChanged: function() {
console.log("ClientChanged");
ready = true;
this._query();
},
sort: {numReviews: "desc"},
aggs: {
category: {
terms: {
field: "category",
order: {
_count: "desc"
}
}
},
location: {
terms: {
field: "location",
order: {
_count: "desc"
}
}
},
numReviews: {
terms: {
field: "numReviews",
order: {
_count: "desc"
}
}
},
polarity: {
terms: {
field: "polarity",
order: {
_count: "desc"
}
}
}
}
}
}).then(function (resp) {
var myids = []
resp.hits.hits.forEach(function(entry){myids.push(entry._id)})
that.ids = myids;
//console.log(that.ids)
that.data = resp;
//console.log(that.data)
});
},
_filtersChange: function() {
this._query();
},
_query: function() {
console.log("_filtersChangedash")
var that = this;
if(filtered){
var client = new $.es.Client({
hosts: this.host
});
client.search({
console.log("Ready?: ", ready);
if(ready){
this.client.search({
// undocumented params are appended to the query string
index: "tourpedia",
type: "places",
......
demo/imports.html 0 → 100644
View file @ cba5d026
<link rel="import" href="../bower_components/polymer/polymer.html">
<link rel="import" href="../bower_components/elastic-client/elastic-client.html">
<link rel="import" href="../bower_components/dashboard-tourpedia/dashboard-tourpedia.html">
demo/index.html 0 → 100644
View file @ cba5d026
<html>
<head>
<script src="../bower_components/webcomponentsjs/webcomponents-lite.js"></script>
<link rel="import" href="imports.html"></link>
</head>
<body>
<template is="dom-bind">
<elastic-client
config='{"host": "http://localhost:9200"}'
client="{{client}}"
cluster-status="{{myStatus}}">
</elastic-client>
<dashboard-tourpedia
client="{{client}}"></dashboard-tourpedia>
<!-- <button id="databutton" onclick="changedata()">Click to change data</button> -->
<script>
var datas =[
{"hits": {
"total": 20000
},
"aggregations": {
"category": {
"buckets": [
{"key": "myObject", "doc_count": 3000},
{"key": "otherObject", "doc_count": 1000}
]
}
}
},
{"hits": {
"total": 30000
},
"aggregations": {
"category": {
"buckets": [
{"key": "myObject", "doc_count": 1000},
{"key": "otherObject", "doc_count": 4000}
]
}
}
}];
var numdata = 0;
<!-- var nc1 = document.getElementById('demo-chart1'); -->
<!-- var nc2 = document.getElementById('demo-chart2'); -->
<!-- nc1.data = nc2.data = datas[0]; -->
<!-- function changedata(){ -->
<!-- numdata += 1; -->
<!-- nc1.data = nc2.data = datas[numdata%2]; -->
<!-- } -->
</script>
</template>
</body>
</html>
docker-compose.yml 0 → 100644
View file @ cba5d026
version: '2'
services:
sefarad:
build: .
ports:
- "8080:8080"
volumes:
- .:/usr/src/app
networks:
- sefarad-network
depends_on:
- elasticsearch
elasticsearch:
image: elasticsearch
ports:
- "9200:9200"
- "9300:9300"
volumes:
- ./elasticsearch/nodes:/usr/share/elasticsearch/data/nodes
- ./elasticsearch/config:/usr/share/elasticsearch/config
networks:
- sefarad-network
luigi:
build:
context: luigi/
volumes:
- ./luigi:/usr/src/app
networks:
- sefarad-network
networks:
sefarad-network:
driver: bridge
\ No newline at end of file
elasticsearch/config/elasticsearch.yml 0 → 100644
View file @ cba5d026
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please see the documentation for further information on configuration options:
# <https://www.elastic.co/guide/en/elasticsearch/reference/5.0/settings.html>
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: sefarad
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#network.host: 192.168.0.1
#
# Set a custom port for HTTP: