Difference between revisions of "Signal DSL"

Revision as of 14:34, 9 November 2018

Purpose

POS master data updates over signal watchers, to execute a list of tasks (for now only import and/or export via datainterchange) depending on newly created files. This so called trigger-file will be deleted after the execution of all the tasks (or actions) weither they are be executed sequentially or parallel.

A client wishes to export a certain amount of data (sales, reports, master data...) on a specific working day (or not) at a certain time, either once or several time on this date. Therefore, it should now be possible to define such characteristic inside a model.

SignalDSL

The SingalDSL defines the java functions for Os.bee and the functions will be used in other Os.bee models. The main semantic elements of the SingalDSL are:

• package - The root element that contains all the other elements. (now in one file only one package could be defined, the grammar could be altered to support several packages in one file, if needed)
• import declarations - Used to import external models or other Java classes.
• watcher - define a watcher which monitors a directory.
• filemask - define a group of target files to watch and all detail about the watcher.
• filename - define a target file to watch and detail about the watcher.
• scheduler - define the simple or complex schedulers for executing tens, hundreds, or even tens-of-thousands of jobs.

package defintion

With the keyword package followed by a qualified name, you define the root element that contains all the other elements. A model can contain multiple packages.

► Syntax:

package <package name>[{ 
	(
	  watcher <watcher name>
		  filemask <file mask> . . .
		| filename <file name> . . .

	| scheduler <scheduler name> . . .
	)* 
}]

• one or more watcher/scheduler can be defined in the same package.
• 2 types of watcher can be defined, they are file mask watcher and file name watcher.
• The trigger policy defines how the execution of its list of tasks be triggered: either file based (the creation of a file) or scheduled plan based.
  • Currently we only support the export and import functions of existing datainterchange units from the datainterchangeDSL as the possible watcher’s/scheduler’s tasks to be executed.
  • The properties of datainterchangeDSL are saved in the datainterchange configuration files which are generated with the datainterchangeDSL.
  • The datainterchange configuration files are default saved in C:\Users\<user name>\.osbee. Per datainterchange group, a <groupname>Config.xml file will be generated automatically. This file path can be changed in Eclipse--> Preferences--> OSBP Application Configuration -->External Data Source --> Datainterchange Setting.
  • The corresponding watcher's and scheduler's properties defined in SingalDSL will be added into the corresponding datainterchange-group-configuration files after saving the SignalDSL file.
  • In cases the needed datainterchange properties are not available in the configuration file, or the configuration file doesn't exist, the default definition of the properties from the datainterchangeDSL file will be used.
  • Currently it is not possible to monitor changes on a (web)url-based directory (destination). You may however define a watcher with a url-based directory and set schedulers to import data from it.
  • Watcher/scheduler generates watcher/scheduler jobs, they will be stored into a queue and processed by the watcher/scheduler job handler. Keyword sequential and parallel determinate how the jobs will be executed. As same as the literally meaning, Keyword sequential means all tasks/jobs will be sequentially executed; Keyword parallel means all tasks/jobs will be parallel executed.

► Datainterchange Group:

group Brands {
	interchange Brandowner merge deleteFileAfterImport file CSV "C:/datatransfers/BrandOwner.csv" delimiter ";" quoteCharacter """ skipLines 1 path {
		entity Brandowner
	}

	interchange Brandtype merge deleteFileAfterImport file CSV "C:/datatransfers/BrandType.csv" delimiter ";" quoteCharacter """ skipLines 1 path {
		entity Brandtype
	}

	interchange Brand merge deleteFileAfterImport file CSV "C:/datatransfers/Brand.csv" delimiter ";" quoteCharacter """ skipLines 1 path {
		entity Brand lookup {
			for brandtypeid in Brandtype createOn "/csv-set/csv-record" mapFrom "id" mapTo id for brandownerid in Brandowner createOn "/csv-set/csv-record" mapFrom "id" mapTo id
			}
		mapping {
			map brandtypecd to "brandtypecd" map bsin to "bsin" mapBlob bsinmedia to "bsin" extension "jpg" path "C:/POD/brands/brand" mimeType jpg map brandnm to "brandnm" map brandlink to "brandlink" map id to "id"
		} 
	}

	interchange Gtin merge deleteFileAfterImport file CSV "C:/datatransfers/Gtin.csv" delimiter ";" quoteCharacter """ skipLines 1 path {
		entity Gtin lookup {
			for brandid in Brand createOn "/csv-set/csv-record" mapFrom "id" allowNoResult mapTo id
		}
	}

	interchange Pkgtype merge deleteFileAfterImport file CSV "C:/datatransfers/PkgType.csv" delimiter ";" quoteCharacter """ skipLines 1 path {
		entity Pkgtype
	}
}

► BrandsConfig.xml:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
<comment>dataInterchange file URLs</comment>
<entry key="Pkgtype-export">C:/datatransfers/PkgType.csv</entry>
<entry key="Max-Parallel-Threads-Count">3</entry>
<entry key="Brand-export">C:/datatransfers/Brand.csv</entry>
<entry key="Pkgtype-import">C:/datatransfers/PkgType.csv</entry>
<entry key="Brandtype-export">C:/datatransfers/BrandType.csv</entry>
<entry key="Brand-import">C:/datatransfers/Brand.csv</entry>
<entry key="Brandtype-import">C:/datatransfers/BrandType.csv</entry>
<entry key="Brandowner-export">C:/datatransfers/BrandOwner.csv</entry>
<entry key="Gtin-export">C:/datatransfers/Gtin.csv</entry>
<entry key="Brandowner-import">C:/datatransfers/BrandOwner.csv</entry>
<entry key="Gtin-import">C:/datatransfers/Gtin.csv</entry>
</properties>

Watcher

Watcher jobs are generated by a watcher, they will be stored into a queue and processed by the watcher job handler.

► Syntax:

watcher <watcher name>
	filemask <file mask>
		sequential | parallel
		from <DataInterchange Group>
	{
		import|export <DataInterchange Unit> [applyon]
	}

	| 
	filename <file name>
		sequential | parallel
		from <DataInterchangeGroup>
	{
		(import|export <DataInterchange Unit> [applyon])*
	}

• 2 types of watcher can be defined, they are:
  • file mask watcher:

    We provide the possibility to set a file mask to extend the further specify (and identify) on a file name basis, the defined task will be executed when the matched files creations occur in the monitored directory.

  • file name watcher:

    A specified file name can be defined as the target file, and a list of tasks will be executed when the matched file creation occurs in the monitored directory.

► Example:

watcher WithMask filemask "Gtin*.csv" parallel from Brands {
	import Gtin
}

watcher WithName filename "brands.csv" parallel from Brands {
	import Brandowner
	import Brandtype
	import Brand
	import Pkgtype
	import Gtin applyon
	export Brand
}

====Target File====^

A target file is the file which is generated inside the monitored directory, whose filename (+extension) matches the file mask or the file name.

The datainterchange file used for import processes will be deleted at the end of a successful import, if it is explicitly set in its datainterchange definition. See [Datainterchange DSL documentation page] for more information.

In case the target file is at the same time used in datainterchange as the import file, it will be deleted at the end of the execution of the watcher job (parallel or sequential), even if it’s not set in the datainterchange definition.

If the import process is failed, at the end of the import process, the corresponding import file will be renamed with form "FAILEDIMPORT" +< actual date formated >+ "Original file name.LOCKED" (as the definition in the following blog post).

Scheduler

The scheduler functionalities are built upon the Quartz Job Scheduler Framework.

Scheduler jobs are timely executed at run-time based upon their scheduler definition.

► Syntax:

scheduler <scheduler name>
	 <scheduler expression>
	[sequential|parallel]
	from <DataInterchange Group>
{
	(import|export <DataInterchange Unit>)*
}

scheduler expression

You are able to define an hourly, a daily, a weekly or a monthly schedule plan, to set when to execute the list of tasks of a Scheduler. Additionally, you can also be more straightforward and define a more complex schedule plan with cron-expressions.

More information about how to build cron-expressions can be find here.

hourly scheduler

hourlyat <minute>

• The minute entry has to be between 0 and 59.

► Example:

scheduler HourlyJob hourlyat 55 from Brands {
	export Brandowner
}

daily scheduler

dailyat <hour> : <minute>

• The hour entry has to be between 0 and 23.
• The minute entry has to be between 0 and 59.

► Example:

scheduler DailyJob dailyat 18 : 10 from Brands {
	export Gtin
}

weekly scheduler

weeklyon <Day Of Week Enum> at <hour> : <minute>

• Day Of Week Enum is including sunday, monday, tuesday, wednesday, thursday, friday and saturday.
• The hour entry has to be between 0 and 23.
• The minute entry has to be between 0 and 59.

► Example:

scheduler WeeklyJob weeklyon saturday at 6 : 15 from Brands {
	export Brandtype
}

monthly scheduler

monthlyon <day of month> at <hour> : <minute>

• The day of month entry has to be between 1 and 31.
• The hour entry has to be between 0 and 23.
• The minute entry has to be between 0 and 59.

► Example:

scheduler MonthlyJob monthlyon 28 at 20 : 45 from Brands {
	export Brand
}

cron scheduler

cron <cron expression>

• the format of cron expression is defined as a string like <second> <minute> <hour> <day of month> <month> <day of week>.
  • The second entry has to be between 0 and 59.
  • The minute entry has to be between 0 and 59.
  • The hour entry has to be between 0 and 23.
  • The day of month entry has to be between 1 and 31.
  • The month entry has to be between 1 and 12.
  • The day of week entry has to be a value of SUN, MON, TUE, WED, THU, FRI, and SAT.

► Example:

scheduler CronJob cron "0 0/5 * * * ?" from Brands {
	import Brandowner
	import Brand
	export Gtin
}

Copyright Notice

All rights are reserved by Compex Systemhaus GmbH. In particular, duplications, translations, microfilming, saving and processing in electronic systems are protected by copyright. Use of this manual is only authorized with the permission of Compex Systemhaus GmbH. Infringements of the law shall be punished in accordance with civil and penal laws. We have taken utmost care in putting together texts and images. Nevertheless, the possibility of errors cannot be completely ruled out. The Figures and information in this manual are only given as approximations unless expressly indicated as binding. Amendments to the manual due to amendments to the standard software remain reserved. Please note that the latest amendments to the manual can be accessed through our helpdesk at any time. The contractually agreed regulations of the licensing and maintenance of the standard software shall apply with regard to liability for any errors in the documentation. Guarantees, particularly guarantees of quality or durability can only be assumed for the manual insofar as its quality or durability are expressly stipulated as guaranteed. If you would like to make a suggestion, the Compex Team would be very pleased to hear from you.

(c) 2016-2024 Compex Systemhaus GmbH