Automated Data Sync
Upon receiving of survey responses, the system can dispatch JSON data to the specified URL.
Automated Data Sync Setting
1. Access the "Settings" page. There are two ways to access it:
- Click Save and Settings after editing the survey
- Click Settings Home and Settings in the "My Surveys" page
2. There are two methods for data sync:
- For a Single Survey: Turn on Automated Data Sync and Enter the synchronization URL, and then the response data from this survey will be sent as JSON to this URL after configuration.
- For All Surveys: Click on Global data sync and activate this feature. Then, enter the synchronization URL. After configuration, all response data will be sent as JSON to this URL.
3. By default, only valid responses are synchronized. This function can be disabled if required.
4. The default setting does not retry on failure. Enable this option to retry sending data if no response is received within 8 seconds, treating the attempt as a failure.
5. You can access synchronization logs to monitor data synchronization activities.
Format of Response Data
To ensure the speed and success rate of the push, the vast majority of question types will only push the numbers corresponding to the questions and options, and will not push the text content of the questions and options. Here's a breakdown of the data format:
- t1 represents the displayed question number, while q1 represents the internal system question number. Note: t1 may not match q1 if the question's position changes.
- t1_q1_1 for Multiple Textbox and Matrix question types refers to the first input field.
- t1_q1_1_2 for Table question types indicates the first row and the second column.
- For closed-ended question types, like Multiple Choice, Checkboxes, Rating the Choice, and more, the "1" in t1_q1":"1" represents the selected option's sequence.
- For open-ended question types, the answers display exactly as entered by respondents.
- Some question types, such as Multi-level Dropdown, Constant Sum, Slider, Matrix Slider, and more, provide the content of the selected option.
- For Conjoint Analysis, all attributes for a concept are displayed. If the concept has been chosen by a respondent, 'Yes' will be shown; otherwise, 'No' will be displayed.
Architecture Recommendation: Separation of Data Reception and Business Processing
Problem Description
The current architecture couples data reception with business processing, which leads to the following issues:
- When business processing fails, it returns a failure response, which triggers a retry mechanism.
- Retried data also fails, creating a retry loop.
Proposed Solution
Adopt a separated architecture for reception and processing:
1. Data Reception Layer
- Upon receiving data, immediately store it in a temporary data pool.
- Return a success response immediately after successful storage.
2. Business Processing Layer
- Asynchronously retrieve data from the temporary data pool for processing.
- Processing failures do not affect the success response from the reception layer.
Solution Advantages
- Avoid channel blocking: Return success immediately upon reception, preventing channel blockage due to business processing failures.
- Improve fault tolerance: Business processing failures do not affect data reception, allowing independent retry mechanisms.
- Enhance performance: Reception layer responds quickly, while business processing can be performed asynchronously and concurrently.
- Facilitate maintenance: Clear separation of responsibilities enables more precise problem identification.
Important Notes
- Ensure sufficient capacity for the temporary data pool.
- Establish data cleanup mechanisms to prevent data accumulation.
- Monitor the status of the data pool and processing success rate.
- Ensure idempotency of data processing operations.
Summary
By separating data reception from business processing, the system can effectively resolve data channel blocking issues caused by business processing failures, thereby improving overall system stability.
