Hi. I've been looking at Gen3's tools for data management and contributions. Are there any higher-level tools along the lines of AWS Glue Studio that could simplify the process for a non-technical curator of building and running tools for tasks such as creating new data model entries, or creating and automatically running a data model mapping on newly ingested data? Sorry for my naivete if I'm missing something obvious.
Greetings,
I don't think you are missing anything. Gen3 has nothing as sophisticated as AWS Glue Studio. Since you mentioned that you have been looking at Gen3's tools, I assume that you found this document Gen3 - Data Contribution which contains some elements that are amenable to less technical curators.
While there are several GUI based elements and forms available to users, the use of the gen3-client is required for uploading of files or data to a commons. Regardless of the commons, placing /submission as a url suffix endpoint gives users access to projects and subsequently forms to map files to meta data nodes.
For users that are more technically saavy, there are many more options Gen3 - Set up Gen3.
I hope some of these resources help and please feel free to reach back out if there are any other specific tools you are curious about.
Dan Biber
CTDS - PlanX User Services
Thanks Dan, this is helpful to know. And so it seems then there are no web/GUI-based tools for modifying the Data Model, but rather just the Dictionary tool for viewing overview and details of the Data Model?
Gen3 doesn't endorse a specific tools for developing the data model, however, the key thing is that your tool can communicate with the database system you are using (PostGresql is our default) AND that it can produce a compliant JSON dictionary. There would probably need to be some additional programming work to get the dictionary to work with Gen3, but hopefully not too much. 