Intra-organization use cases
An organization wants to create three different accounts due to different audit, logging, and security requirements for other groups. The three accounts are, Production account (PROD), Stage account (STAGE), and Development Account (DEV).
- Production data is shared with STAGE.
- STAGE has more source data loading into the table, validating the data loaded correctly.
- STAGE shares the updated data with DEV.
- DEV needs to modify data by conducting some transformation, and then shares it with PROD.
Based on the requirement, here is the Snowflake Data Sharing implementation.
In this big data era, data sharing is critical for corporate communication and operations across industries and organizations. The ability to make data easily consumable by people who can draw value from it remains rudimentary as data grows in size and volume. The majority of an organization’s data is locked away in numerous silos, with no easy method to make it accessible to potential consumers. Because massive datasets must now move across business borders by wire, the task of allowing data access has become substantially more difficult. As a result, the economic possibility for data sharing will continue to rise, creating unmet market needs for us.
Large datasets often have to be shared, whether between business departments within an enterprise or externally amidst several parties. The fact that Snowflake allows you to share data sets with anyone who has access to Snowflake accounts makes it the ideal cloud platform for data sharing.
Data Sharing in Snowflake enables account-to-account sharing of Snowflake database tables and views. The main participants on each side of a data-sharing relationship are the owner of the accounts at either end of the data-sharing relationship and any individual who shares specific accounts from your single master account with co-workers.
Snowflake Data Sharing is about taking your preferred account data model and exposing just the appropriate attributes across one or multiple accounts owned by separate organizations. That way each account contains the minimum amount of information for its use. No matter where your data is hosted, Snowflake has a solution that allows you to share it seamlessly between groups of people who need access. Simply set up a Data Sharing object within any table in your preferred account and click on the SHARES section in the left menu to start adding collaborators and configuring access privileges. Edit the container object’s metadata to tailor it exactly to your needs; if multiple accounts are involved, you can change the owner at any time!
Although a concept that may take some getting used to, shared data in the Snowflake Cost Table does not consume storage space for consumers. Therefore, Snowflake table storage expenses would not be reflected on your monthly bills. In addition, because no data is copied, arranging and making the most of sharing is easily executed in minimal time, achieving instant results.
Moreover, by enabling data sharing from their Snowflake accounts, a large base of customers can easily be offered access to datasets from data producers. In comparison, companies that purchase data can then easily combine it with other datasets or information for further analysis and refining before utilizing it. For instance, a staffing company may have access to candidate records with related job applications. They could then mix these records and public profiles with their in-house client references about the candidates. This has the potential to make for more accurate referrals while retaining a competitive advantage if the new combined datasets were to remain exclusive.
Please note that “data sharing” in Snowflake is as simple as moving data within your hard disk or between folders. You can already do this without special software, but with Snowflake Data Warehouse, you get to do it on a much larger scale! With the way AWS S3 works, if you have multiple buckets, you can access files from the bucket of any other user so long as those files are shared; with Snowflake, setting up data sharing is similar. It allows users to move tables and their contained objects from one database into another. Similarly, you could move entire databases within objects into different clusters using native tools like BigQuery Transfer Tool because it allows for table transfer of entire BigQuery tables across different projects!
Lyft, shift, and load any data into Snowflake instantly
Lyftrondata makes three offers available to its customers: direct data share, data marketplace, and data exchange. There is no difference in the technical underpinnings between all three offers – their only distinction is that they each have their own unique functionality.
How Lyftrondata helps to transform your Snowflake
Snowflake Integration is taken seriously in Lyftrondata’s efforts to overcome the limitations of current ETL tools that stage flat files on a local disk before pushing them to Snowflake. Such a solution requires extra disk space for data loading and adds additional delays to the data pipeline. Lyftrondata takes a unique approach to data loading to the Snowflake Data Warehouse by utilizing new data streaming capabilities in Snowflake.
Streaming transformation pushdown turns your streaming database into a SQL data warehouse. Once data is loaded, transformations are applied to the streamed records on the fly as they are written into the database. No need to pre-load any data or manage space for temporary files. You can get all the advantages of a SQL data warehouse while still keeping your streaming application deployed continuously.
The use cases given below are some of the most prevalent ones among customers. They are designed to give you ideas on how to develop a data-sharing use case that meets your requirements.
- Production data is shared with STAGE
- STAGE has more source data loading into the table, validating the data loaded correctly.
- STAGE shares the updated data with DEV.
- DEV needs to modify data by adding some transformation, and then shares it with PROD.
- PROD shares its table TB1 to both stages via Share1.
- STAGE creates a new table TBB using CTAS (Create Table As Select from TB1 of PROD)
- STAGE shares TBB to DEV via Share2.
- DEV creates a new table TBD using CTAS (Create Table As Select from TBB of STAGE)
- DEV shares TBD to PROD via Share3.
- PROD switches to using TBD of PROD instead of TB1
- PROD creates a new table TB1_V2 using CTAS (Create Table As Select from TBD of DEV)
- ALTER TABLE TB1 RENAME TO TB1_V1;
- ALTER TABLE TB1_V2 RENAME TO TB1;
How Lyftrondata drives powerful BI acceleration
Lyftrondata’s data-sharing framework is built upon the secured data governance technology so that users with approved access can access the shared data in real-time without any movement of the original data. With this secured framework, you can create instant insights as well as additional business value while improving your data governance.
Lyftrondata’s modern architecture provides row and column level access security to users making it easier, and more secure than ever before, to share your data without writing a single piece of code. With Lyftrondata’s robust and secure framework, you can enable governance on your data hub and easily maintain one-to-one, one-to-many, and many-to-many relationships with your customers & partners.
Let’s get personal: See Lyftrondata on your data in a live demo
