Material Big Data

Lanzados ppts informativos de tecnologías BigData: Hadoop, Hbase, Hive, Zookeeper...

Apuntate al Curso de PowerBI. Totalmente práctico, aprende los principales trucos con los mejores especialistas

Imprescindible para el mercado laboral actual. Con Certificado de realización!!

Pentaho Analytics. Un gran salto

Ya se ha lanzado Pentaho 8 y con grandes sorpresas. Descubre con nosotros las mejoras de la mejor suite Open BI

LinceBI, la mejor solución Big Data Analytics basada en Open Source

LinceBI incluye Reports, OLAP, Dashboards, Scorecards, Machine Learning y Big Data. Pruébala!!

23 may. 2019

List of Open Source solutions for Smart Cities - Internet of Things projects


Increasingly projects are carried on so-called 'Smart Cities', supported by Big Data, Internet of Things... and the good news is that most of them are made with Open Source technologies. We can share, from TodoBI.com our insights about these technologies

Making a city “smart” involves a set of areas we will outline below: Without IOT (Internet Of Things), there will be no Smart City. 

Since automatic collected data is the most efficient way to get huge amounts of information, devices connected to the internet are an essential part of a Smart City.
The way we store and process data from city is generally using Big Data and Real Time Streaming technologies. 

The final goal where more innovative and custom analysis can be achieved using Artificial Intelligence and Machine Learning. Finally I would include Apps, as usually this kind of solutions is consumed in mobile devices. 




Here we outline the common process of building a Smart City solution: 

-Choose data 
-Connecting devices 
-Design Data Storage Infrastructure 
-Real Time Events and Notifications 
-Analytics -Visualization (Dashboards) 

 1) Choosing Data 

In a city there are three basic sources of data: citizens, systems, sensors. Use the available information of users, on social networks, informations systems, public statistical information offered by the administration. 

A typical example is user with geolocalization enabled in twitter. Information about the systems and services in a city are sometimes available in open data sources. An example could be the water or electricity consumption. 

Last but not least, sensors. A city hoping to become “Smart” has to intend to provide automatic information of its environment, and that could be achieved using sensors. Sensors can be anywhere


2) Connecting Devices


Devices (sensors) connects with the real time data streaming and the storage infrastructure using efficient communications protocols, that using light weight packaging and asynchronous communications.


Examples of some communications protocols used:


MQTT (Message Queuing Telemetry Transport) Websocket (bi-directional web communication and connection management)


STOMP (The Simple Text Oriented Messaging Protocol)


XMPP (Extensible Messaging and Presence Protocol)



3) Design Data Storage Infraestructure 


The Data Storage Infrastructure for a Smart City solutions has special characteristics, due to the diversity and dynamism of its sources. 


Time series DB are frequently used, because of the time evolution of data catched by sensors Some examples of this kind of DB are InfluxDB and Druid

Another DB commonly used in Smart Cities project are MongoDB (json format advantages), Cassandra (fast insertion advantages), Hadoop (big data frameworks advantages)

Some samples





4) Real Time events and notifications


Usually Smart Cities solutions have needs for real time notifications on events. To accomplish such requirements the system must have a Stream Analytic engine, that can react to events in real time and send notification. This characteristics bring us some technologies related to this; Storm, Spark Streaming, Flink, WebSocket, Socket.IO



IoT Frameworks:



Node-RED



Node-RED is a tool for wiring together hardware devices, APIs and online services in new and interesting ways.

The light-weight runtime is built on Node.js, taking full advantage of its event-driven, non-blocking model. This makes it ideal to run at the edge of the network on low-cost hardware such as the Raspberry Pi as well as in the cloud.

The flows created in Node-RED are stored using JSON which can be easily imported and exported for sharing with others.
An online flow library allows you to share your best flows with the world


PubNub




     PubNub is a Data Stream Network, that offers infrastructure as a service. With PubNub,  we can use the infrastructure provided and connect our devices, designing our architecture and simply get advantages of all this.

PubNub has 5 main tools:

-Publish Subscribe (Allows Real Time Notifications of Events to users)
-Stream Controller (Allows managing channels and groups of channels)
-Presence (Allows notifications when users login or leave the system, or similar behaviour, device availability for example)
-Access Manager (Allows administrators, to grant or deny permitson users of the systems)
-Storage & Playback (Provide storage for messages,and allows messages retrieval at later time)



IoT-AWS



AWS IoT is a platform that enables you to connect devices to AWS Services and other devices, secure data and interactions, process and act upon device data, and enable applications to interact with devices even when they are offline


5) Analytics and Visualization



You can show real time dashboards, reports, OLAP Analysis using tools like Pentaho. See samples of Analytics  

15 comentarios:

Lalita Chaple dijo...

Thanks for sharing information about Internet of Things.
Internet of Things Technology and Solutions

Unknown dijo...

Very interesting,good job and thanks for sharing such a good blog.
Crafsol Connect to all with connectivity and get generate data from the Internet of Things(IoT)
Internet of Things Solutions

Marc dijo...

Hi. I would like to ask you what do you think about microsoft dynamics 365. Could you recommend me this solutions? Your opinion is very valuable for me.c

bahadur kaun dijo...

Thanks for updating this quality stuff. Treasurebox always provide you quality stuff for your home and garden.
We will also provide you outdoor furniture with home delievery.

BERITA OLAHRAGA dijo...

Nice Article ,,, Come and Join our website to guys
BOLA PELANGI - BANDAR MIX PARLAY TERBESAR DI INDONESIA HANYA DENGAN 1 USER ID SUDAH BISA BERMAIN SEMUA GAME.
Join US ! klik link di bawah ini ☟
Prediksi Bola dan Togel.
Bola Online
Mix Parlay
Bola Jalan
Bandar Bola

Marysia dijo...


It's very interesting. I think that the company's development is very important. Especially from the very beginning. I regret that I did not do it from the beginning. I think it would be easier now. Well, but there is nothing to think back, it's best to look ahead :)
I'm now thinking about cloud solutions ( read it more). Seriously, this is very interesting.

Shanthi Cabs dijo...

The article is very interesting and very understood to be read, may be useful for the people. I wanted to thank you for this great read!! I definitely enjoyed every little bit of it. I have to bookmarked to check out new stuff on your post. Thanks for sharing the information keep updating, looking forward for more posts..
Kindly visit us @
Madurai Travels | Travels in Madurai
Best Travels in Madurai
Cabs in Madurai | Madurai Cabs
Tours and Travels in Madurai

Rigid Box dijo...

Excellent Blog. I really want to admire the quality of this post. I like the way of your presentation of ideas, views and valuable content. No doubt you are doing great work. I’ll be waiting for your next post. Thanks .Keep it up!
Kindly visit us @
Luxury Packaging Box
Wallet Box
Perfume Box Manufacturer
Candle Packaging Boxes
Luxury Leather Box
Luxury Clothes Box
Luxury Cosmetics Box
Shoe Box Manufacturer
Luxury Watch Box

KIT dijo...

Nice blog, it's so knowledgeable, informative, and good looking site. I appreciate your hard work. Good job. Thank you for this wonderful sharing with us. Keep Sharing.
Kindly visit us @
100% Job Placement
Best Colleges for Computer Engineering
Biomedical Engineering Colleges in Coimbatore
Best Biotechnology Colleges in Tamilnadu
Biotechnology Colleges in Coimbatore
Biotechnology Courses in Coimbatore
Best MCA Colleges in Tamilnadu
Best MBA Colleges in Coimbatore
Engineering Courses in Tamilnadu
Engg Colleges in Coimbatore

ASIABET4D dijo...

Kami Asiabet4D Agen Judi Bola Online yang telah lama berkerja sama dengan agen judi bola online dan telah mempunyai ribuan member aktif dan setia Taruhan Bola Online thanks you bruhh.

You are invited to our website too..
Agen Judi Bola Online www.play.asiabet4d.id
Best Regards,

TULISTOTO OFFICIAL dijo...


Why can you make this good article? I hope you can also give input to me for this brilliant idea at Agen Casino Online Terpercaya I'm waiting bro.
Perisakti Prediksi Togel dan Bola Paling Jitu

Jackie Co Kad dijo...

Great Article
Final Year Project Domains for CSE
Final Year Project Centers in Chennai





JavaScript Training in Chennai
JavaScript Training in Chennai

baby NAgaQQ dijo...

NAGAQQ | AGEN BANDARQ | BANDARQ ONLINE | ADUQ ONLINE | DOMINOQQ TERBAIK

Yang Merupakan AGEN BANDARQ TERBAIK, Domino 99, Dan Bandar Poker Online Terpercaya di asia hadir untuk anda semua dengan permainan permainan menarik dan bonus menarik untuk anda semua

Bonus yang diberikan NagaQQ :
* Bonus rollingan 0.5%,setiap senin di bagikannya
* Bonus Refferal 10% + 10%,seumur hidup
* Bonus Jackpot, yang dapat anda dapatkan dengan mudah
* Minimal Depo 15.000
* Minimal WD 20.000

Memegang Gelar atau title sebagai AGEN BANDARQ Terbaik di masanya

Games Yang di Hadirkan NagaQQ :
* Poker Online
* BandarQ
* Domino99
* Bandar Poker
* Bandar66(NEW GAMES)

Info Lebih lanjut Kunjungi :
Website : NAGAQQ
WHATSAPP : +855967014811
Line : Cs_nagaQQ
TELEGRAM :+855967014811

INFO NAGAQQ



BACA JUGA BLOGSPORT KAMI YANG LAIN:
berita nagaqq/
agen bandarq online/
Kemenangan NagaQQ/

Edward dijo...

Thank you so much for this nice information. Hope so many people will get aware of this and useful as well. And please keep update like this.

Big Data Services

Data Lake Services

Advanced Analytics Services

Full Stack Development Services

ROBO backup for business dijo...

Thanks for updates on Internet of Things!