on 08-08-2014 8:30 AM
Hello, I am new to ESP and recently doing some researching of this product. I currently have two concept questions, please have a look.
I realize that a Window could save events. My question is, for a derived window which has query defined on it, what kind of data is exactly it will save? are the incoming events (maybe from two or more different previous window or stream) of this window will be saved first and then apply the query and then output; or, it will apply query first and then save the result of the query into the Window and then output?
Thanks
Hi,
A, quick answer, it does not save incomming events only what is needed by the query and the retention policy.
It is often described as a data-flow maried with a sql like language.
The clauses or queries acts as filters and only the records are saved or stored in the window that matches the query. Those records are not saved indefinitely normally, but as long as it fullfils the keep-clause, i.e. you can keep them for 5 minutes or you can keep 500 rows possibly per column value.
The rows are propagated using insert, update etc. data events as you can see for example when you manually feed in some data in the studio or look at some splash code examples like the Top3TradesFlex in the Splash Programming chapter.
Thanks,
Ben
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Plujim,
Thanks for the reply. Let me use an example to describe my question. In this developer center we have a sample about capture the Weather station and the weather data. In that example, we have a derived window and the query of it is to join the weather data with the station master data. Both weather data and station is the input of this window. Now my question would be, are the result (the joined data between weather data and station) will be saved into this window, or the input (weather data and station) will be saved into this window?
Thanks
It might be useful to add a bit more detail here:
- As Ben describes, the output of the window is the result of the query logic attached to that window.
- What's held in that window is not the input events but the results of the continuous query, where these results are updated each time the window receive a new input event
- The KEEP clause affects what's in the window, and the behavior of the KEEP clause is different depending on where it's used:
CREATE LOCAL WINDOW RecentWeatherData
PRIMARY KEY DEDUCED
KEEP 4 HOURS
AS SELECT
...
Then this tells the window to only keep the rows in the window that have been added or updated on the last 4 hours. It doesn't affect the inputs to the window, just the rows "maintained" in the window (i.e. the output of the window)
CREATE OUTPUT WINDOW AVG_TEMP
PRIMARY KEY DEDUCED
AS SELECT ...
FROM EVENTS KEEP 30 SEC
GROUP BY EVENTS.MACHINEID ;
Would have the effect of only computing the group values across the set of input events from "EVENTS" that arrived in the last 30 seconds
Here are a couple of posts that might be useful:
Creating windows from event streams: http://scn.sap.com/docs/DOC-40921
Understanding opcodes: http://scn.sap.com/docs/DOC-40207
Hi Jeff,
Many thanks for your complementary. Please allow me ask further questions: for this example,
CREATE OUTPUT WINDOW AVG_TEMP
PRIMARY KEY DEDUCED
AS SELECT ...
FROM EVENTS KEEP 30 SEC
GROUP BY EVENTS.MACHINEID ;
do you mean the system will first create an unnamed window which save the query result (SELECT XXX FROM EVENTS) for 30 seconds, and then the system will create your named window based on that unnamed window , do the group by aggregation and save the group by result? if so, in principle, can I also set KEEP on the header of the CREATE WINDOW statement as well to set the retain policy for the group by result? ( so in this one statement we could have two KEEP policies, on for NAMED WINDOW and one for UNNAMED WINDOW? )
Not quite. The system will create an unnamed 30 second window on the stream "EVENTS", holding all events received in the last 30 seconds. The query (SELECT...GROUP BY) will be run against that 30 second window.
Yes, you can also set a KEEP policy on AVG_TEMP, eg:
CREATE OUTPUT WINDOW AVG_TEMP
PRIMARY KEY DEDUCED
KEEP 5 MINUTES
AS SELECT ...
This will not affect the 30 second window that serves as the input to the continuous query. What it will do is automatically remove any summary row from AVG_TEMP for a group that has not be create or updated in the last 5 minutes.
User | Count |
---|---|
84 | |
23 | |
11 | |
9 | |
8 | |
5 | |
5 | |
5 | |
5 | |
4 |
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.