Material Big Data

Lanzados ppts informativos de tecnologías BigData: Hadoop, Hbase, Hive, Zookeeper...

Apuntate al Curso de PowerBI. Totalmente práctico, aprende los principales trucos con los mejores especialistas

Imprescindible para el mercado laboral actual. Con Certificado de realización!!

Pentaho Analytics. Un gran salto

Ya se ha lanzado Pentaho 8 y con grandes sorpresas. Descubre con nosotros las mejoras de la mejor suite Open BI

Aprende gratis Analytics OLAP sobre Pentaho

La solución open source para business intelligence y Big Data sobre Pentaho, no te lo pierdas!!

30 oct. 2017

Workshop gratuito BI Open Source Pentaho en Lima, Peru (21 de Noviembre)


Os presentamos un más que interesante Workshop gratuito de Pentaho en Lima, Perú. Será realizado por los especialistas de Stratebi

PUBLICO OBJETIVO

Para todos los que se quieran dedicar al mundo del Business Intelligence, profesionales de las tecnologías de información, gestores de TI, consultores en Business Intelligence, Analistas de Negocio, Analistas de sistemas, arquitectos Java, desarrolladores de sistemas, administradores de bases de datos, desarrolladores y profesionales con relación a el área de tecnología, marketing, negocio y financiera.
Si desean inscribirse o  formación 'online completa', adaptada a sus necesidades, pueden contactar con info@stratebi.com

OBJETIVO

El objetivo es enseñar al alumno las posibilidades para construir una solución de Business Intelligence (BI) para hacer el análisis de datos procedentes de diversas fuentes y sistemas, utilizando herramientas de software libre como Pentaho. Herramienta líder en el mercado Open Source.
También se hablará sobre otros entornos BI Open Source como Saiku, Ctools, Talend y otras soluciones desarrolladas por la comunidad.

29 oct. 2017

Project Maestro: ETL para Tableau




Que duda cabe, como indicábamos hace unos días en la comparativa entre Tableau y PowerBI que uno de los elementos que se achaca como carencia a Tableau es la ausencia de herramienta de ETL.

Lo cierto es que lleva bastante tiempo anunciando su propia herramienta ETL, Project Maestro, más de un año, aunque por lo que conocemos, se asemejaría más a un módulo de Data Preparation orientado a usuario final que una herramienta ETL completa. 

En nuestra opinión y práctica diaria, buena parte de compañías que usan Tableau y PowerBI, que tienen necesidades de ETL importantes, se decantan por el uso de Pentaho Data Integration y Talend Open Studio, para orquestar todos sus procesos

En cualquier caso, la iniciativa de Tableau es interesante para aquellas compañías/usuarios que no tengan necesidades importantes en cuanto a ETL y quieran hacerlas ellos mismos directamente



La cuestión... es que hay que seguir esperando... de momento


26 oct. 2017

New features in STDashboard for Pentaho



The improvements in this version of STDashboard are focused on user interface for panel and dashboard and also some enhancement in performance and close some old bugs. It works with Pentaho versions 5, 6 and 7

You can see it in action in this Pentaho Demo Online





About UI improvements:

 - New set of predefined dashboard templates. We have designed a new way to manage dashboard panels that allow you to shape the dashboard in almost any combination of size, proportion and amount of panel you want to have. For this reason we have created a set of different layouts for most common cases.



 - Self managed panel. Add and remove panels, now in stdashboard you can add or remove panels easily using the button inside each panel header.



 - New layout management. Now an stashboard layout is composed of a list panel container, the containers in this list are stacked vertically in the page. There are two types of such containers; horizontal and vertical, each one stores a list of real panels (the ones where the graph are drawn) in an horizontal or vertical flow, in this ways you can combine those panels to achieve almost any layout you can imagine.



 - Resizable panels. We have included the possibility of resize the panel horizontally or vertically, keeping the proportion of graph inside it in correspondence with horizontal adjacent panels without making an horizontal scroll in the page, that means if you shrink a panel horizontally and there is another panel in the same row, the other panels also shrink an a proportional way to allow all panels in a row fit the horizontal size of the window. 

Is interesting to note here that we have implemented this functionality using pure GWT API, to avoid external dependencies and ensure portability between browsers.

 - Draggable panels. Each panel in the entire dashboard can be dragged to any parent container. In the header of each single panel the is a handle that allow dragging the panels to any panel container in the dashboard.




 - Responsive Dashboard. The ability to resize dynamically the panels and graph when the window's dimensions change, or when a user make zoom in the page is now implemented, also in most phones the dashboard can be seen proportionally and keeping the original layout.

 - Persistent state of the layout. When you save a dashboard to a file, we are saving the visual state of it and store it in the file. Then, when you open the dashboard, all the details of visual interface are hold and you can see the dashboard exactly the same previous to saved, that means panels size, locations are restored effectively.


About performance:

 - In some points of the application an specific query was causing performance problem. To know if a member has child or not in a multilevel hierarchy, the previous code issued a query to list all the sons of that member and check if the size is greater than 0, our solutions in this case for this type of query was simply check the level of the current member and in this way answer that boolean query.

 - Connection to cubes using the new MondrianOlap4jDriver java class. This improve the connection performance and stability because is designed for mondrian connections, the previous code was using an standard JDBC connection.


About new enhacements:

- Date configuration for filters. Date dimension are special dimensions, because almost any cube has at least one defined and are very used for make range query over fact table, to allow dynamic filter in panels, we had to enable a .property file that allow the user to define their date dimension and configure the way they want to use it in queries.


Added the Pentaho File Explorer to allows the users navigation through the files stored in pentaho, like reports, documents, etc and embeed it inside a panel in the dashboard







See a Video Demo:

24 oct. 2017

Fintech radar en España, una nueva burbuja?



Que duda cabe, que el surgimiento de iniciativas tecnológicas alrededor del campo de las finanzas, Fintech, en España, puede considerarse una buena noticia.

No obstante, viendo experiencias pasadas de burbujas 2.0 y anteriores en el año 2.000, hay que se cautos; para separar el grano de la paja. De una bonita idea, logo, oficinas, etc... a su aplicación práctica con retorno de la inversión, puede distar mucho

En cualquier caso, la lista esta disponible, :-)

More than 2600 Open Data Portals around the world


The table of contents will give you a summary of all countries represented on this list. Simply click on a country’s name and the page will bring you to the correct section.

If you are curious about how we created this list, an article about it, thanks to OpenDataSoft

18 oct. 2017

Human Resources Analytics


Human Resources LinceBI Analytics solution is based on open source including KPIs, Reports, OLAP Analysis, Dashboards, Scorecards, Big Data and Machine Learning with 'predefined templates, dashboards and KPIs/ratios and fully customizable environment

Manage budgets efficiently and maximize revenues and costs in favour of collective benefit.



Do more with less! Through innovative techniques of Data Mining and Social Intelligence to maximize objectives, identifying trends related to workers behavior and satisfaction in order to answer their demands efficiently and improve engagement










15 oct. 2017

Comparativa de Costes Tableau vs PowerBI

 

Os dejamos un documento listo para descargar, con una comparativa muy completa de costes entre Tableau y PowerBI (hay que decir que el informe ha sido encargado por Tableau, por lo que puede tener cierto sesgo). 

Por ejemplo, en cuanto al esfuerzo de este tipo de proyectos, si tenemos en cuenta que ambas son herramientas de Data Discovery (usuario final), no se tiene suficientemente en cuenta la parte más importante, el modelado, ETL, Data Quality, etc... 

En la práctica, estas herramientas, necesitan también de herramientas ETL, metadatos, MDM, Data Quality que garanticen la correcta implementación en entornos en producción

Para una comparativa de funcionalidades técnicas echad un vistazo a la Comparativa de herramientas Business Intelligence

Ver también: Como preparar un entorno Big Data OLAP con Tableau y con PowerBI








Use Case “Dashboard with Kylin (OLAP Hadoop) & Power BI”



In recent posts, we explained how to fill the gap between Big Data and OLAP, using Tableau, Pentaho and Apache Zeppelin.

Now, we´ll show you how to use PowerBI for Big Data Dashboards using Apache Kylin. Also try online in our Big Data Demo site


Arquitecture:
In this use case we have used together Apache Kylin and Power BI to support interactive data analysis (OLAP) and developing a dashboard, from data source with Big Data features (Volume, Speed, Variety).


The data source contains the last 15 years of academic data from a big university. Over this data source, we have designed a multidimensional model with the aim of analyze student’s academic performance. We have stored in our Data Warehouse about 100 million rows, with metrics like credits, passed subjects, etc. The analysis of these facts is based on dimensions like gender, qualification, date, time or academic year.
However this data volume is too large to be analyzed using traditional database systems for OLAP interactive analysis. To address this issue, we decide to try Apache Kylin, a new technology that promises sub second interactive queries for data Volumes over billions and trillion of rows on the fact table.
Apache Kylin architecture is based on two Hadoop stack technologies: Apache Hive and HBase. First, we have to implement the Data Warehouse (DW) on Hive database using a star or a snow flake schemas. Once we have implemented one of these data models, we can define an OLAP cube on Kylin. 
To this end, we have also to define a Kylin’s cube model using Kylin’s GUI with wizard. At this moment, Kylin can generate the MOLAP cube in an automatic process. After cube creation, we can query the OLAP cube using SQL queries or connecting to a BI tool using the available J/ODBC connectors.
With aim to explore the data and generate visualizations that allows users to extract useful knowledge from data, we have chosen Microsoft Power BI tools: Power BI Desktop and Power BI Service (free of charge version).
Power BI Desktop is a completely free desktop self-service BI tool that enable users to create professional dashboards easily, dragging and dropping data concepts and charts to a new dashboard. Using this tool we have developed a dashboard, similar to our use cases with Tableau or Apache Zeppelin.
Once designed the dashboard, we have published it on the Web with Power BI cloud service (free edition). In other to do that, we have to create an extract of the data and upload it with the dashboard.  This process is transparent to users, who also can program data refreshing frequency using Pro or Premium versions of the Power BI service (commercial tools).


Apache Kylin:


Developed by eBay and later released as Apache Open Source Project, Kylin is an open source analytical middle ware that supports the support analysis OLAP of big volumes of information with Big Data charactertistics, (Volume, Speed, and Variety).
But nevertheless, until Kylin appeared in the market, OLAP technologies was limited to Relational Databases, or in some cases optimized for multidimensional storage, with serious limitations on Big Data.
Apache Kylin, builded on top of many technologies of Hadoop environment, offer an SQL interface that allows querying data set for multidimensional analysis, achieving response time of a few seconds, over 10 millios rows.
There are keys technologies for Kylin; Apache Hive and Apache HBase
The Data Warehouse is based on a Start Model stored on Apache Hive. 
Using this model and a definition of a meta-data model, Kylin builds a multidimensional MOLAP Cube in HBase. 
After the cube is builded the users can query it, using an SQL based language with its JDBC driver.
When Kylin receives an SQL query, decide if it can be resolved using the MOLAP cube in HBase (in milliseconds), or not, in this case Kylin build its own query and execute it in the Apache Hive Storage, this case is rarely used.
As Kylin has a JDBC driver, we can connect it, to most popular BI tools, like Tableau, or any framework that uses JDBC.

PowerBI:



Power BI is a set of Business Intelligence (BI) tools created by Microsoft. Due to its simplicity and powerful, this emerging tools are becoming a leader BI technology like others such as Tableau, Pentaho or Microstrategy. 
Like these technologies, Power BI is a self-service BI tool, extremely simple but with a lot of powerful features as the following: dashboard developing (called reports in Power BI), web and intra organization sharing and collaborative work, including dozens of powerful charts (ej. line chart with forecasting on page 2 of our demo), connection to relational and Big Data sources, support for natural language Q & A, support to execute and visualize R statistic programs or data preprocessing (ETL).
The above features are implemented across the different tools of Power BI suite. Power BI desktop is a desktop tool for data discovery, transformation and visualization. It is a completely free tool with connectors to the most used relational and Big Data sources. Although for same data sources there are specific connectors, with Apache Kylin we have to use the ODBC connector available on Apache Kylin web page. In this way, we connect to Kylin and a data extract from data source is automatically generated by Power BI. 
At this moment we can create our demo visualization as follows: i) define data model, ii), apply some data transformations if needed (e.g. date format), iii) generate calculated metrics (e.g. student success rate), and then, iv), create the dashboard visualization, with one or multiple pages (e.g. our demo has two page interchangeable with bottom bar selector).
At this time, we have used Power BI service (cloud) to publish on the web our new dashboard join with data extract. To this end, we created an account of Power BI free. In this case, there are also Pro and Premium commercial editions with additional features like data extraction automatic refreshing and direct connections to some data sources such as SQL Server (also Analysis Services), Oracle or Cloudera Impala. 
However none of these direct connectors are for Apache Kylin, then with Kylin we have to use data extraction and data extract refreshing approaches.  
In addition to Power BI Desktop and Power BI Services (Free, Pro and Premium) there are other Power BI tools such as Power BI Mobile (access to dashboard from smartphone and collaborative work) or Power BI Embedded (to use visualizations in ad-hoc apps, web portals, etc).

If you are interested to implement your BI company project with Power BI do not hesitate to contact us on StrateBI.


Open Source Business Intelligence tips in October 2017

10 oct. 2017

Pentaho 8 Reporting for Java Developers


Gracias a Packt que nos ha enviado una copia de: 'Pentaho 8 Reporting for Java Developers' para revisión, como hemos hecho en otras ocasiones y que publicaremos proximamente

Este libro está escrito por un buen amigo con el que hemos coincidido en bastantes Pentaho Developers, Francesco Corti. Echad un vistazo a su web, gran experto en Alfresco y su integración con Pentaho.

Más de 400 páginas de utilidad en este libro, con código para ejercicios

Puedes ver también, el tutorial gratuito sobre Pentaho

5 oct. 2017

Cuales son las novedades es MySQL 8.0?



MySQL, the popular open-source database that’s a standard element in many web application stacks, has unveiled the first release candidate for version 8.0.
Features to be rolled out in MySQL 8.0 include:
  • First-class support for Unicode 9.0 out of the box.
  • Window functions and recursive SQL syntax, for queries that previously weren’t possible or would have been difficult to write.
  • Expanded support for native JSON data and document-store functionality.
With version 8.0, MySQL is jumping several versions in its numbering (from 5.5), due to 6.0 being nixed and 7.0 being reserved for the clustering version of MySQL.

MySQL 8.0’s expected release date

MySQL hasn’t committed to a release date for MySQL 8.0, by MySQL’s policy is “a new [general] release every 18-24 months.” The last general release was October 21, 2015, for MySQL 5.7, so MySQL 8.0’s production version is likely to come in October 2017

Where to download MySQL 8.0
You can download the beta versions of MySQL 8.0 now for Windows, MacOS, several versions of Linux, FreeBSD, and Solaris; the source code is also available. Scroll down the downloads page and go to the Development Releases tab to get them.

Visto en Infoworld

1 oct. 2017

Google lanza Cloud Dataprep in public beta



Muy interesante esta iniciativa de Google en Cloud, Cloud Dataprep, con la idea de facilitar los procesos ETL. Os dejamos la info más abajo, pero según nuestra opinión, dos temas importantes a considerar:

- Data preparation es un eufemismo para intentar dar a entender que los procesos ETL pueden ser sencillos y para usuarios finales, algo que para cualquiera que se dedique al Analytics sabe que no lo es (de hecho, es la parte más compleja e importante, es como la parte oculta de un iceberg). Y, esto es, por que se vislumbra mercado/ingresos en este área. Ver siguiente punto:

- Tiene un modelo de pricing

Google Cloud Dataprep is an intelligent data service for visually exploring, cleaning, and preparing structured and unstructured data for analysis. Cloud Dataprep is serverless and works at any scale. There is no infrastructure to deploy or manage. Easy data preparation with clicks and no code.

The stories behind the data



No dejéis de echar un vistazo a esta iniciativa de Bill Gates: The stories behind the data

"We are launching this report this year and will publish it every year until 2030 because we want to accelerate progress in the fight against poverty by helping to diagnose urgent problems, identify promising solutions, measure and interpret key results, and spread best practices.
As it happens, this report comes out at a time when there is more doubt than usual about the world’s commitment to development."

Microsoft lanza nuevas herramientas de Machine Learning



Microsoft, just like many of its competitors, has gone all in on machine learning. That emphasis is on full display at the company’s Ignite conference this where, where the company today announced a number of new tools for developers who want to build new A.I. models and users who simply want to make use of these pre-existing models — either from their own teams or from Microsoft.

For developers, the company launched three major new tools today: the Azure Machine Learning Experimentation service, the Azure Machine Learning Workbench and the Azure Machine Learning Model Management service.


In addition, Microsoft also launched a new set of tools for developers who want to use its Visual Studio Code IDE for building models with CNTK, TensorFlow, Theano, Keras and Caffe2. And for non-developers, Microsoft is also bringing Azure-based machine learning models to Excel users, who will now be able to call up the AI functions that their company’s data scientists have created right from their spreadsheets.

Visto en Techcrunch

Te puede interesar: Las 53 claves para conocer Machine Learning