[GIS] How to write all attributes when multiple source files have the same name

fields-attributesfmemapinfospatial-database

I am using FME to write all the MapInfo TAB files in a directory and it's subdirectories to a File Geodatabase. I'm using dynamic readers and dynamic writers as I don't know the geometry or schema of the input files beforehand. So far so good. Now, suppose that I have a file called "Pipeline.TAB" at the root of the directory and I have another file called "Pipeline.TAB" in a subdirectory. No problem. FME can merge the two files together and I get the geometry from both files in my new feature class called "Pipline_line".

My issue is, suppose those two original files have different attributes. Say, the file in the root folder has a field called "Name" and the second file has a field called "Pipe_Name". My resulting feature class gets the "Name" field but not the "Pipe_Name" field. The features that came from the file in the subdirectory have a NULL value in the "Name" field. Instead, what I would like is to have both fields come through. I'm happy if the "Name" field is NULL as long as the "Pipe_Name" field is there and has the correct values.

It appears that the schema is determined by the first feature that is being read, whereas, I need it to use the schema from all of the features with the same name. Is this possible?

EDIT

I have posted this question on FMEPedia as well. Here is some additional information which may help:

The example I gave was simple in that it only had two files called Pipeline.TAB. In reality, I'm looking at dozens of folders with thousands of files. there might be 20 files called "Pipeline" and another twenty called "Valves", etc. I won't know the schema of all of them at the start.

What I would like the final attribute table of my destination feature class to look like would be something like this:

Name       | Pipe_Name
Pipe1      | NULL
Pipe2      | NULL
Pipe3      | NULL
NULL       | Pipe 4
NULL       | Pipe 5

In the final file, there may actually be many fields. But I won't know all of them at the start:

Name      | Pipe_Name  | Diameter | Material | Type
Pipe1     | NULL       | 500      | Steel    | Gas
Pipe2     | NULL       | 1200     | Steel    | Gas
Pipe3     | NULL       | 800      | PVC      | Water
NULL      | Pipe 4     | NULL     |NULL      | Water
NULL      | Pipe 5     | NULL     |NULL      | Water

In response to @BradNesom's answer, I have looked at the SchemaMapper, but I have way too many files to look at for that to be feasible. I also have looked at the other link you mentioned and I tried adding a reader as a resource but that didn't seem to work either.

Best Answer

Yes it is possible.
How would be the rest of your question.
Using the schema mapper transformer is a great direction but if you have "lots" of different files with no good way of cataloging all the variety.
In this fmepedia article there are multiple examples.
I like the attributerenamer method.
But there is another that allows the schema to be read at run time.
Specifically, the Dynamic Schema functionality provides two options for determining the destination schema information at run-time:
from a reader already in the workspace (Source Reader),
from a reader added as a Workspace Resource (Workspace Resource Reader).
EDIT:
I just found the collectors section on fme store.

featuremerger

UnconditionalFeatureMerger

This transformer merges sets of attributes onto all REQUESTOR features. Essentially it is an unconditional merge of feature attributes.
The user can choose between two modes:
One Supplier, or All Suppliers.

In One Supplier mode,
All REQUESTORS will receive a copy of the attributes of the first SUPPLIER feature,
and emerge from the COMPLETE output port.
The first SUPPLIER emerges from the REFERENCED output port; any others through the
UNREFERENCED port.

In All Suppliers mode,
All REQUESTORS will receive a copy of the attributes of all SUPPLIER features, and emerge from the COMPLETE output port. All SUPPLIER features emerge from the REFERENCED output port.

Related Question