cancel
Showing results for 
Search instead for 
Did you mean: 

Question on a basic concept of ESP Window

richard_gu
Explorer
0 Kudos

Hello, I am new to ESP and recently doing some researching  of this product. I currently have two concept questions, please have a look.

I realize that a Window could save events. My question is, for a derived window which has query defined on it, what kind of data is exactly it will save? are the incoming events (maybe from two or more different previous window or stream) of this window will be saved first and then apply the query and then output; or, it will apply query first and then save the result of the query into the Window and then output?

Thanks

Accepted Solutions (1)

Accepted Solutions (1)

Former Member
0 Kudos

Hi,

A, quick answer, it does not save incomming events only what is needed by the query and the retention policy.

It is often described as a data-flow maried with a sql like language.

The clauses or queries acts as filters and only the records are saved or stored in the window that matches the query. Those records are not saved indefinitely normally, but as long as it fullfils the keep-clause, i.e. you can keep them for 5 minutes or you can keep 500 rows possibly per column value.

The rows are propagated using insert, update etc. data events as you can see for example when you manually feed in some data in the studio or look at some splash code examples like the Top3TradesFlex in the Splash Programming chapter.

Thanks,

Ben

richard_gu
Explorer
0 Kudos

Hi Plujim,

Thanks for the reply. Let me use an example to describe my question. In this developer center we have a sample about capture the Weather station and the weather data. In that example,  we have a derived window and the query of it is to join the weather data with the station master data. Both weather data and station is the input of this window. Now my question would be, are the result (the joined data between weather data and station) will be saved into this window, or the input (weather data and station) will be saved into this window?

Thanks

Former Member
0 Kudos

Hi,

The derived window should contain the joined data and not the input of both weather data and station. If you have some kind of aggregration I think the events opcode lead to modification of the result (as a delta); a sum or otherwise will not be recalculated from scratch.

Ben

richard_gu
Explorer
0 Kudos

So, how a window would be CRUD are mainly depends on the opCodes of output, right?

Thanks indeed

Former Member
0 Kudos

Thinks so , yes,

Ben

JWootton
Advisor
Advisor
0 Kudos

It might be useful to add a bit more detail here:

- As Ben describes,  the output of the window is the result of the query logic attached to that window.

- What's held in that window is not the input events but the results of the continuous query, where these results are updated each time the window receive a new input event

- The KEEP clause affects what's in the window, and the behavior of the KEEP clause is different depending on where it's used:

  • When you apply the KEEP clause to the Window itself, like this:

CREATE LOCAL WINDOW RecentWeatherData

PRIMARY KEY DEDUCED

KEEP 4 HOURS

AS SELECT

...

Then this tells the window to only keep the rows in the window that have been added or updated on the last 4 hours.  It doesn't affect the inputs to the window, just the rows "maintained" in the window (i.e. the output of the window)

  • When you apply the KEEP clause as part of the FROM clause, it creates a an unnamed window - think of it as an invisible window - on the input to the window (or stream) being created by the CCL CREATE statement that contains the FROM clause.  So this:

CREATE OUTPUT WINDOW AVG_TEMP

PRIMARY KEY DEDUCED

AS SELECT ...

FROM EVENTS KEEP 30 SEC

GROUP BY EVENTS.MACHINEID ;


Would have the effect of only computing the group values across the set of input events from "EVENTS" that arrived in the last 30 seconds

Here are a couple of posts that might be useful:

Creating windows from event streams:  http://scn.sap.com/docs/DOC-40921

Understanding opcodes:  http://scn.sap.com/docs/DOC-40207

richard_gu
Explorer
0 Kudos

Hi Jeff,

Many thanks for your complementary. Please allow me ask further questions: for this example,

CREATE OUTPUT WINDOW AVG_TEMP

PRIMARY KEY DEDUCED

AS SELECT ...

FROM EVENTS KEEP 30 SEC

GROUP BY EVENTS.MACHINEID ;

do you mean the system will first create an unnamed window which save the query result (SELECT XXX FROM EVENTS) for 30 seconds, and then the system will create your named window based on that unnamed window , do the group by aggregation and save the group by result? if so, in principle, can I also set KEEP on the header of the CREATE WINDOW statement as well to set the retain policy for the group by result? ( so in this one statement we could have two KEEP policies, on for NAMED WINDOW and one for UNNAMED WINDOW? )

JWootton
Advisor
Advisor
0 Kudos

Not quite.  The system will create an unnamed 30 second window on the stream "EVENTS", holding all events received in the last 30 seconds.  The query (SELECT...GROUP BY) will be run against that 30 second window.

Yes, you can also set a KEEP policy on AVG_TEMP, eg:

CREATE OUTPUT WINDOW AVG_TEMP

PRIMARY KEY DEDUCED

KEEP 5 MINUTES

AS SELECT ...

This will not affect the 30 second window that serves as the input to the continuous query.  What it will do is automatically remove any summary row from AVG_TEMP for a group that has not be create or updated in the last 5 minutes.

Answers (0)