This project has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant agreement no 609043 EU flag

Information about EVENTFLOWS

How it works

How it works- Get Started


Marketplace System Description

Front End

The EventFlows Marketplace has been created and configured including categorization of available artefacts (such as data streams, services and events), ability to filter on various criteria such as offering type, location or specific fields that may exist in each offering type (e.g. QoS levels, technology for getting data etc). Users may create free accounts, browse through the available events and register to one or more of them. They may also create their own event (details on this to follow). Link to your Paypal account may be performed for the artefacts that are sold and for which payment is needed.

Back End Eventflows Messaging Platform

Eventflows comes with a backend common messaging platform, based on RabbitMQ, in order to push notifications between producers and consumers, for the artefacts of Data Streams and Events. Upon creation of a listing, the producer will be sent information (user credentials, endpoint etc) on which they need to push their produced data. Similarly a consumer, upon registration to an event, will be sent credentials in order to access the messaging system and be able to consume the provided information.
One key aspect is that the system to have consistency in terms of namespaces and management, these users (both producers and consumers) have limited rights for configuration. Therefore
widely used Node-RED nodes that could be used for accessing RabbitMQ (based on the AMQP protocol) like https://www.npmjs.com/package/node-red-contrib-amqp can not be used. The reason for this is that the specific implementation requires extensive rights in order to be able to operate, such as configuration rights for creation of exchanges and queues. However in the marketplace pub/sub system, these rights are reserved only for the administrator role and not for the producer or consumer role. For this reason we created a set of customized Node-RED flows that may be able to override this issue and can be found hereThe flow consists of a main function incorporating the node.js function of amqplib, appropriately configured to be used with the Eventflows messaging system. However a set of configuration steps are also needed, as indicated in the README file included in the flow, which includes inserting e.g. credentials acquired, exchange name and url of the messaging system, as provided by the marketplace.

What is Eventflows and what it is NOT

The Eventflows Marketplace IS:
  • A common discovery point, enabling you to publicise and describe  your work
  • A handling point for your registration and billing schemes
  • A common messaging system that may accomodate distribution of the produced data between producers and consumers
  • A starting point for you to get involved by reusing flows and concepts around Smart Cities and IoT/Service technologies
The Eventflows Marketplace IS NOT:
A service hosting platform. The implementations needed for generating an artefact (event, service or data stream) should reside at the producer side or in a relevant Cloud environment (e.g. IBM Bluemix or similar)



Artefact Consumer and Producer Guidelines

Producer Guidelines Per Artefact Type

As an Artefact producer, in order to provide it through Eventflows, you need initially to create an account. From there you can post a new listing specifying the artefact (Data Stream, Service or Event) and adding specific information with relation to it. Necessary information differs per listing type but in general it includes the description of the artefact, the definition of the pricing scheme and more importantly the JSON template (and an example data item) through which event information is published. Then you need to utilize the provided client in order to push the event notifications to the Eventflows messaging system, through which it will be circulated to the subscribed consumers. Payments are performed automatically via Paypal upon registration of a consumer to your artefact.
What is critical for you to consider and include during the description is for which audience/market your artefact is expected to be useful, given that this is also one of the search criteria for consumers and may be indicative for the usage of a given artefact. Thus it may boost the consumption of your artefact. You can even go one step ahead and if you have in mind a specific artefact (with a given target audience), before implementing it spend some time doing a small survey to the related audience in order to validate whether the specific artefact would be actually useful.
Furthermore, for all cases of artefacts, their actual production resides at the producer's side and only the produced information is distributed through the Eventflows messaging system.

Data Streams
If you are a data stream producer, then it is most likely that you will be exploiting a primary open data source that you may manipulate in order to produce some added value (e.g. format change, aggregation, transformation from pull to push model, annotation of the stream with additional information etc) then the most important requirement for your case is the investigation of the license that may accompany the usage of your  input data. DIstributing data from other sources is not always legal or needs a specific action (e.g. acknowledgement of the source) and you are responsible for abiding by the wishes of the original data producer.
Furthermore, for designing an added value data source, that will be enriched with relation to the original one, you may need to combine it with an added value service (described below) in order to create demand around your artefact. As an example, getting the raw data from Twitter streams is meaningless, since someone might do it directly, but providing an enriched data flow that uses an e.g. Sentiment Analysis service to populate each tweet datum with the sentiment category could be useful to consumers.

Added Value Services
The key difference for this type of service is that at the moment the Eventlfows Messaging System does not include the incorporation of service calls. Therefore you should also describe the endpoint and API through which you will expose your services.
When exposing services, the guidelines are as follows:
  • Consider that you need to expose services that perform a generic operation that needs to be clearly described and provide a key added value. As examples we include getting a text string as argument and returning its sentiment, classified in specific categories, making predictions for the fluctuation of a given value (e.g. stock exchange variations) etc.
  • Your service implementation should be able to accomodate multi-user requests, distinguishing potentially based on different configurations, if applicable by the service scope
  • Your service's API should be described somewhere (a relevant description field for including the API URI is included in the Service Listing type)
  • At the moment pricing plans supported by Eventflows are per request amount, but the actual calls do not pass through the Eventflows platform. This means that while registrations and payment may be performed through Eventflows, monitoring  and limitation of usage per consumer should be performed at the service side.

Event streams
In order to support developers in their event production, Eventflows has a list of potential technologies and means that can be used for enhancing the process, derived from the outcomes of the EU funded FP7 COSMOS project. We have found that Node-RED (an open source graphical environment based on node.js) is optimal for purposes similar to the ones of an Event Producer, hence we suggest its usage even for the creation of the flow that will generate the event. However this may be performed in whatever manner you see fit or you are accustomed to, provided that you eventually push the information to the Eventflows Messaging Platform in the defined manner. In general, some useful guidelines follow:

  • The first thing that you need to consider is what type of event notification you need to implement. Combining different data sources, services and technologies such as analytics may lead to a more meaningful event, but you should always have in mind to whom you are appealing to and what kind of added value your event will give them. This should be treated also as key input with relation to who your customers will be, how much they are willing to pay for a specific event and how you can reach them when you are trying to adverise your work (oh yes, Eventflows will promote your work but you should do too in your channels!)
  • Do consider aspects of your business model before deciding how much you want to charge your custormers. Generating an event might involve costs from your side, such as needed Cloud servers etc. Therefore we recommend that you initially build a prototype flow that generates the event in low cost infrastructures (e.g. your own local server), list the event in the marketplace and only if necessary you can consider going to the next step of a more production environment like IBM Bluemix. By doing that you may also check some key assumptions that you may have performed when investigating a specific event, e.g. how many registrations you expect  
  • The format of the event should be in JSON. However the content of the JSON (definition of fields etc.) is up to you. The only requirement in this case is that you have described the JSON schema to be used during the listing creation for the event notification.
  • Once the event notification technical process is created at your side, you need to create a new listing in Eventflows that describes the event you are notifying about, along with a number of technical fields. More information on the listing creation process can be found here.
  • Using the AMQP protocol you should push the data item in the respective endpoint designated by Eventflows upon registration

In order to support Producers, we have developed a set of flows, in the context of COSMOS, that may help you get started in your event production! More information can be found in the related Github repository!


Consumer Guidelines per Artefact Type

Data Streams and Events
As an Artefact consumer, you need again to create an account and subscribe for consuming an artefact, abiding to the offering scheme of the producer. When a subscription has been performed you will receive a client and credentials for logging into the Eventflows messaging system, from which you will be able to register to the specific endpoint of the artefact (Event or Data Stream) in which you are interested. After you receive the event stream, you may perform any kind of local processing at your side, such as forwarding the event to a mobile app or even using it to produce a new event as mentioned below. Instructions for registering and acquiring information is included here, while the Node-RED clients are included here.

Added Value Services
At the moment, consumption of Added Value Services is not performed via the Eventlfows Messaging System but directly through the service endpoint defined by the producer. In each service offering, these details should be described (along with the API specification of the calls to be made).

Circular Business Model Considerations
It is implied that the roles of producer and consumer may be interchangeable in the context of the Marketplace. Thus a consumer of Events A and B may combine these events (or with another data source) and create a more finegrained Event C that is again published through the Marketplace. However in this case you should also take under consideration the baseline costs of consuming Events A and B for determining your pricing for Event C.

Request an artefact that is offered for free

For an artefact that is offered for free (Giving away), you can consume it without going through the payment process. For that to happen you need to send a message to the Eventflows team (from the Menu tab go to Contact us tab and use the box to indicate which listing you want). Then Eventflows will send in your designated email the credentials for accessing the specific listing content.

Request an artefact that is not currently provided

If you, as a Consumer, do not find a needed artefact, or you need a modified one based on your case (e.g. Tweet feed for Madrid exists but you need it for London), you can always suggest its creation in the Producer Community.
For this to happen we have two ways of doing it:
  • You can create a new issue, labeled as ENHANCEMENT!, in the community github account, with your comments that may include also differences with the way an existing artefact is produced at the moment
  • You can create a new listing, marked as "Requested", in the Eventflows Marketplace front end, where you may also utilize the existing fields per category in order to define how you would like this artefact to be exposed