11 Replies Latest reply: Feb 28, 2011 4:37 AM by 841426 RSS

    Reading files from the internet and writing them to a computer

    802236
      Hi,

      I am working on a program to copy files from a location on the internet to a local file on a computer.
      The file is a .swf file, so I read the data from the file on the internet into a ByteBuffer and then write it to a file on the computer.....
      But, it isn't working......It copies the file, but the .swf doesn't run right
      However, when I use mostly the same process to copy a file from a local file to another local file on the same computer, it copies just fine.....
      As far as I know I am copying files in both of these circumstances the same way, with just a few changes to connect to a URL...

      I can post source code in a day or two if that would help explain better...

      Any help would be greatly appreciated!!!
        • 1. Re: Reading files from the internet and writing them to a computer
          jtahlborn
          no one can help you without seeing your code.
          • 2. Re: Reading files from the internet and writing them to a computer
            796440
            ClaymoreBoom wrote:
            Hi,

            I am working on a program to copy files from a location on the internet to a local file on a computer.
            The file is a .swf file, so I read the data from the file on the internet into a ByteBuffer and then write it to a file on the computer.....
            But, it isn't working......It copies the file, but the .swf doesn't run right
            There's a bug in your code.
            I can post source code in a day or two if that would help explain better...
            "If"? Did you honestly consider it a possibility that the above description would be enough for somebody to diagnose your problem?
            • 3. Re: Reading files from the internet and writing them to a computer
              baftos
              Point your browser to a file on the internet. Not necessarily swf, but some file that the browser does not know how to render (ex.zip). Download it.
              Now download again with your java code.
              Are the two files exactly the same? (use command line 'comp' command if in Windows).
              If not, your code is wrong without even seeing it.
              • 4. Re: Reading files from the internet and writing them to a computer
                802236
                Ok, here is the source code for my program that copies files from an internet source:
                import java.io.*;
                import java.net.URL;
                import java.net.URLConnection;
                 
                public class ReadWrite {
                 
                     public static void main(String[] args) {
                          
                          byte outText[] = LoadFromUrl("http://taprootconnections.com/flash/flash_slide_show2.swf");
                          //byte outText[] = LoadFromFile("C://tool2.swf");
                          String strFilePath = "C://FileIO//tool.swf";
                      
                           try
                           {
                            FileOutputStream fos = new FileOutputStream(strFilePath);
                           
                            fos.write(outText);
                      
                           fos.close(); 
                      
                          }
                          catch(FileNotFoundException ex)
                          {
                           System.out.println("FileNotFoundException : " + ex);
                          }
                          catch(IOException ioe)
                          {
                               System.out.println("IOException : " + ioe);
                          }
                     }
                     
                     static byte[] LoadFromFile(String source)
                     {
                          File file = new File(source);
                          byte fileContent[] = null;
                          try {
                             FileInputStream fin = new FileInputStream(file);
                           
                            fileContent = new byte[(int)file.length()];
                
                            fin.read(fileContent);
                      
                          } catch (IOException err) {
                              System.err.println(err);
                          }
                          return fileContent;
                     }
                
                     static byte[] LoadFromUrl(String source)
                     {
                          byte fileContent[] = null;
                          URLConnection con = null;
                          try{
                               URL url = new URL(source);
                               con  = url.openConnection();
                               InputStream in = con.getInputStream();
                               
                               fileContent = new byte[(int)con.getContentLength()];
                
                              in.read(fileContent);
                               
                          } catch (IOException err) {
                              System.err.println(err);
                          } 
                          return fileContent;
                
                     }
                }
                I tried comparing two files, one downloaded with a web browser, and the other with my program, and they don't equal each other.....

                However, I did a similar test with files on my hard drive, just copying them locally by switching the comments that declare outText. There is no problem with copying files locally, the problem only appears with getting a file from a URL.

                I apologize for posting before without the source code handy....
                But I really appreciate any help with this..... I just don't see why it isn't working. Thanks!
                • 5. Re: Reading files from the internet and writing them to a computer
                  796440
                  in.read(byte[]) is not guaranteed to fill the array. You need to read in a loop until it returns -1.

                  At the very least, check the return value of that read() call. If it does not say that you've read the full file, then there's your problem. If it says you did read the full file, then there's a different issue that's not immediately obvious to me. I just latched on to this one because it jumped out at me as an obvious no-no.
                  • 6. Re: Reading files from the internet and writing them to a computer
                    EJP
                    You shouldn't pass around entire files in memory at all. I would completely redesign that method not to return the byte[] but to do whatever you need to do with it internally. If you're writing to a disk you could pass in the output stream as a parameter and just copy the correct way, as jverd points out:
                    byte[] buffer = new byte[8192];
                    int count;
                    while ((count = in.read(buffer)) > 0)
                      out.write(buffer, 0, count);
                    • 7. Re: Reading files from the internet and writing them to a computer
                      802236
                      Thanks for your help,

                      I got it working, and jverd was mostly right about the in.read(byte[]) method. There was one other insignificant issue that I solved, and no worries EJP because now I refactored it into a nice method. I just pass in the source and the destination and it works perfectly.

                      Thanks again, without your suggestions I might not have realized the "right" way to read files.
                      • 8. Re: Reading files from the internet and writing them to a computer
                        EJP
                        jverd was mostly right
                        What do you mean 'mostly'? He was completely correct about that.
                        • 9. Re: Reading files from the internet and writing them to a computer
                          796440
                          EJP wrote:
                          jverd was mostly right
                          What do you mean 'mostly'? He was completely correct about that.
                          Well, I did just focus on the immediate question and ignore the bigger picture that he shouldn't read the whole file into memory. Of course, that's one of the advantages of a forum like this as opposed to the one-on-one correspondence that some posters seek--if one respondent misses something, there are others around to keep him honest.
                          • 10. Re: Reading files from the internet and writing them to a computer
                            802236
                            jverd wrote:
                            EJP wrote:
                            jverd was mostly right
                            What do you mean 'mostly'? He was completely correct about that.
                            Well, I did just focus on the immediate question and ignore the bigger picture that he shouldn't read the whole file into memory. Of course, that's one of the advantages of a forum like this as opposed to the one-on-one correspondence that some posters seek--if one respondent misses something, there are others around to keep him honest.
                            Ya, that is what I meant, jverd was completely right about in.read(byte[])....
                            And thanks again for the help!
                            • 11. Re: Reading files from the internet and writing them to a computer
                              841426
                              If you are willing to try jdk7 then you could use the Files.copy method:
                              http://download.java.net/jdk7/docs/api/java/nio/file/Files.html#copy%28java.io.InputStream,%20java.nio.file.Path,%20java.nio.file.CopyOption...%29