Clarity

  • 1.  Script to move file stored in database to Clarity Filestore

    Posted Apr 13, 2016 02:26 AM

    Hi,

     

    We are in process to Replace existing standalone system with an alternative solution with Clarity. As part of this we need to move existing records from old system to clarity. The old system used to save documents attached any instances into database but with Clarity we use file-store to store attachments to store documents belongs to any object instances. Now I need to write gel script that retrieve the file stored in table column and FTP it to file-store location.

    I got the script to FTP file from local folder to remote folder but I need script to retrieve the file from database to store in local folder.

    Any sample script or suggestion to achieve the goal.

     

    Thanks,

    Shiva



  • 2.  Re: Script to move file stored in database to Clarity Filestore

    Posted Apr 13, 2016 04:04 AM

    If you XOG your files into Clarity (once you have pulled them out of "old system"), if Clarity is set up using a file-system file store then the files will end up on the file-system.

     

    I don't think that you should be attempting to move the files to the Clarity file-system outside of Clarity.

     

    Depending on the volumes though, it might be easier to do manually ; extract from "old system", manually upload to Clarity - if I have say < 100 documents I'd be tempted to do it manually rather than worry about the build/dev/test/prove/run effort in building some XOG solution.



  • 3.  Re: Script to move file stored in database to Clarity Filestore

    Posted Apr 13, 2016 05:09 AM

    There are more than 5K records in old system. Manual work is not possible.



  • 4.  Re: Script to move file stored in database to Clarity Filestore

    Posted Apr 13, 2016 05:13 AM

    Fair enough.

     

    But your original question "I need script to retrieve the file from database to store in local folder" - surely that has nothing to do with Clarity and everything to do with "old system" ; not sure what help you expect here for that?



  • 5.  Re: Script to move file stored in database to Clarity Filestore

    Posted Apr 13, 2016 05:45 AM

    I mentioned that because the FTP tag has local folder. Thought the files has to be in local folder before FTP.

    <ftp:put fileName="data.csv"    localDir="c:/temp" remoteDir="/tmp"/>  </ftp:open>



  • 6.  Re: Script to move file stored in database to Clarity Filestore

    Posted Apr 15, 2016 04:37 AM

    Hello Shiva,

     

    we did migration opposite direction (documents stored in CA Clarity PPM database -> to another system). We proceeded as follows: Export files from CA Clarity DB to application server folder structure (there were also scripts to give files real names and so on, but it's not important now) -> transfer files to temporary storage accessible by new system -> run script in new system to move files to the right folder structure.

     

    So I think, that you should run some "export" script in your old system to export files from DB and then run script in Clarity to move files to the right folder structure. The second option I can think of is that if the old system provides a webservice to export files from database, than you can use script only in Clarity to call the webservice to export files from old system database and than move them to the right structure. But we do not know your old system so we can talk about it only in general, I highly doubt that there is a general script to export file from DB (I might be wrong).

     

    Regards,

    Martin



  • 7.  Re: Script to move file stored in database to Clarity Filestore

    Posted May 03, 2016 06:17 AM

    Thanks Martin,

     

    I can get any web service to extract files from old system DB to local folder but the real problem is to attach the files to particular instances that they belongs to.With clarity you can xog-in the files but it will be huge xog script and we may have to do it in batches. This is why I was looking for any other possibilities to attach files to instance.



  • 8.  Re: Script to move file stored in database to Clarity Filestore
    Best Answer

    Posted May 31, 2016 07:44 AM

    Here is the SP that I have developed to extract the files from database to local folder.

     

    ALTER PROCEDURE [Z_EXTRACT_EACE_DOCUMENTS]
    AS
    BEGIN
      SET NOCOUNT ON;
    
    
      DECLARE @V_PATH VARCHAR(200)
      ,@V_OPATH VARCHAR(200)
      ,@OBJECTTOKEN INT
      ,@V_DOCUMENT_ID NUMERIC
      ,@V_FILENAME VARCHAR(200)
      ,@V_FILE VARBINARY(MAX)
      ,@V_DIRECTORY VARCHAR(200)
      ,@V_FOLDER_RESULT INT
      ,@V_FILE_RESULT INT
      ,@V_Folder_PATH VARCHAR(200)
      ,@V_FILE_PATH VARCHAR(200)
      ,@V_FOLDER_CHECK INT
    
    
      /* Fetch file details to cursor */
      BEGIN
      DECLARE CR_ACE_DOC CURSOR
      FOR
      SELECT attachmentID,filename,filedata
      FROM dbo.tblAttachments ace
      WHERE ace.attachmentID NOT IN (
      SELECT attachment_id
      FROM dbo.z_file_extract_status
      WHERE attachment_id = ace.attachmentid
      AND file_cerated = 1
      )
      END
    
    
      /* Open cursor */
      BEGIN
      OPEN CR_ACE_DOC
    
    
      FETCH NEXT
      FROM CR_ACE_DOC
      INTO @V_DOCUMENT_ID
      ,@V_FILENAME
      ,@V_FILE
    
    
      WHILE @@FETCH_STATUS = 0
      BEGIN
      /* Create directory to extract eace attachment */
      SET @V_PATH = '\\clarity01\docs\' + CONVERT(VARCHAR, @V_DOCUMENT_ID)
      SET @V_DIRECTORY = 'mkdir ' + @V_PATH
    
    
      /* Assign folder creation status to field  */
      EXEC @V_FOLDER_RESULT = master..xp_cmdshell @V_DIRECTORY, no_output
    
      /* check if folder is already created  */
      SET @V_FOLDER_PATH = 'dir ' + @V_PATH + ' /B'
      EXEC @V_FOLDER_CHECK = master..xp_cmdshell @V_FOLDER_PATH, no_output
    
    
      IF (@V_FOLDER_RESULT = 0 or @V_FOLDER_CHECK = 0 )
      BEGIN
      IF (@V_FOLDER_CHECK = 0)
      BEGIN
      /* delete record in log table if folder is already created and dont had files */
      delete from z_file_extract_status where attachment_id = @V_DOCUMENT_ID
      END
    
      /* insert into log table */
      INSERT INTO z_file_extract_status (attachment_id,folder_created,file_cerated,Comments)
      VALUES (@V_DOCUMENT_ID,1,NULL,'Folder created Successfully')
    
    
      /* Extract the document to newly created folder */
      SET @V_OPATH = @V_PATH + '\' + @V_FILENAME
    
    
      EXEC sp_OACreate 'ADODB.Stream',@ObjectToken OUTPUT
      EXEC sp_OASetProperty @ObjectToken,'Type',1
      EXEC sp_OAMethod @ObjectToken,'Open'
      EXEC sp_OAMethod @ObjectToken,'Write',NULL,@V_FILE
      EXEC @V_FILE_RESULT = sp_OAMethod @ObjectToken,'SaveToFile',NULL,@V_OPATH,2
      EXEC sp_OAMethod @ObjectToken,'Close'
      EXEC sp_OADestroy @ObjectToken
    
      IF (@V_FILE_RESULT = 0)
      BEGIN
      /* update log table on file export */
      update z_file_extract_status set folder_created = 1
      ,file_cerated = 1
      ,Comments = 'Folder files are exported Successfully'
      where attachment_id = @V_DOCUMENT_ID
      END
      ELSE
      BEGIN
      /* insert into log table */
      update z_file_extract_status set folder_created = 1
      ,file_cerated = 0
      ,Comments = 'Folder created but files creation failed'
      where attachment_id = @V_DOCUMENT_ID
      END
    
      END
      ELSE
      BEGIN
    
      /* insert into log table */
      INSERT INTO z_file_extract_status (attachment_id,folder_created,file_cerated,Comments)
      VALUES (@V_DOCUMENT_ID,0,NULL,'Folder creation failed')
    
      END
    
    
      FETCH NEXT
      FROM CR_ACE_DOC
      INTO @V_DOCUMENT_ID
      ,@V_FILENAME
      ,@V_FILE
      END
    
    
      /* Close and Deallocate cursor */
      CLOSE CR_ACE_DOC
    
    
      DEALLOCATE CR_ACE_DOC
      END
    
    
      SET NOCOUNT OFF;
    END
    

     

    Regards,

    Shiva



  • 9.  Re: Script to move file stored in database to Clarity Filestore

    Posted Feb 09, 2018 10:22 AM

    Hi Shiva,

     

    We are doing a merge of two different clarity instances, one clarity have documents in filestore and one has in Database. i am trying to migrate the document sin DB to to filestore.

     

    in the above procedure, Could you please tell me what this table refers to... dbo.tblAttachments this table is not part of clarity schema.

     

    appreciate any help regarding the same.



  • 10.  Re: Script to move file stored in database to Clarity Filestore

    Posted Apr 04, 2018 12:07 AM

    Hi,

     

    It is not in clarity schema. It was the table from other application database from where we get the files and moved into clarity through XOG.

     

    Thanks,

    Shiva