Jdbc_static dealing with parameters from not existing fields

Elastic Stack Logstash
1

I'm trying to use the jdbc_static filter in my logstash pipeline, but I am getting the following error:

[2025-04-29T10:36:48,983][WARN ][logstash.filters.jdbc.lookup][test][udplogstash] Parameter field not found in event {:lookup_id=>"loadingdata", :invalid_parameters=>["[account]", "[chaincode]"]}

My lookups is

		local_lookups => [
			{
				id => "loadingdata"
				query => "select ServiceName, ProcessorPartner from localstore where RNID = :RNID and ChainCode = :CHAINCODE"
				parameters => { 
								RNID => "[account]"
								CHAINCODE => "[chaincode]" 
							}
				target => "attributes"
				default_hash => {
					ServiceName => nil
					ProcessorPartner => nil
				}
			}
		]

I know that account and chaincode could not be in the event but I supposed that the query would treat missing fields as null.

Plus, my account and chaincode come in as strings, even if they contain and integer, and I need to cast them to integer.

How would you modify my lookup to pass this problem?

2

If the parameter fields do not exist then it logs a warning and does not do the lookup. (See here and here.)

If this is the only lookup then you could wrap the jdbc_static filter in a test to see of the fields exist:

if [account] and [chaincode] { jdbc_static {...} }

If you want to suppress all warnings from the filter you could configure log4j to do that.

You can use mutate+convert to modify the fields to be integers.

3

@Badger if I want to execute multiple lookups on the same table, conditioned if a field exists, how would I do? Can I have the "if" inside jdbc_static?

4

You cannot have a conditional inside the filter, you have to use conditionals to select which filters to run.

1 Like