ToToTEK.COM Forum Index ToToTEK.COM
Help & Support Forum
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Super UFO PRO 8 SD - USB File transfer alternatives?
Goto page 1, 2, 3  Next
 
Post new topic   Reply to topic    ToToTEK.COM Forum Index -> Copiers and Hardware
View previous topic :: View next topic  
Author Message
Trenton_net



Joined: 29 Jun 2006
Posts: 233

PostPosted: Thu Dec 17, 2015 3:27 pm    Post subject: Super UFO PRO 8 SD - USB File transfer alternatives? Reply with quote

Hey Everyone,

I know the Super UFO Pro 8 SD copier has a USB slot for loading games with from a PC. However, the software included is pretty barebone with no real command-line / non GUI method to invoke it. Does anyone know if the transfer protocol has been figured out? Ie. support could be built into ucon64 or something else of that kind? I don't really have the skills to monitor the USB port, sniff, and reverse the protocol myself unfortunately.
Back to top
View user's profile Send private message
conceitedjerk



Joined: 04 Jan 2004
Posts: 69
Location: Cardboard box behind Sears

PostPosted: Fri Apr 06, 2018 9:50 pm    Post subject: Reply with quote

An acquaintance of mine on Mastodon was doing some work in this regard:

https://gist.github.com/devinacker/72d107bcc18235ef4ef112538ebc72b5
Back to top
View user's profile Send private message Visit poster's website
dbjh



Joined: 02 Aug 2003
Posts: 167

PostPosted: Thu Apr 12, 2018 1:39 pm    Post subject: Re: Super UFO PRO 8 SD - USB File transfer alternatives? Reply with quote

Trenton_net wrote:
Hey Everyone,

I know the Super UFO Pro 8 SD copier has a USB slot for loading games with from a PC. However, the software included is pretty barebone with no real command-line / non GUI method to invoke it. Does anyone know if the transfer protocol has been figured out? Ie. support could be built into ucon64 or something else of that kind? I don't really have the skills to monitor the USB port, sniff, and reverse the protocol myself unfortunately.

Just out of curiosity, why are you interested in transferring files through a USB cable? Does it offer any benefit over copying the files to an SD card directly? Can the UFO SD even fully function with just a USB cable plugged in?
Perhaps I will buy a UFO SD in the not so distant future, because they are quite cheap. However, I will not spend any time on reverse engineering the communication protocol, because it would be a major task for me and I don't see the benefit. I would of course happily accept a patch that would add that functionality to uCON64.

conceitedjerk wrote:
An acquaintance of mine on Mastodon was doing some work in this regard:

https://gist.github.com/devinacker/72d107bcc18235ef4ef112538ebc72b5

Thanks for the link. It does appear to conflict with my findings when dumping cartridges:
https://sourceforge.net/p/ucon64/svn/HEAD/tree/trunk/ucon64/src/backup/ufosd.h
Perhaps that is just the difference between dumping and the effect of writing to certain registers. That would be incredibly sloppy though.
Back to top
View user's profile Send private message
Trenton_net



Joined: 29 Jun 2006
Posts: 233

PostPosted: Wed May 16, 2018 2:42 pm    Post subject: Reply with quote

@dbjh: Hey. Thanks for the kind reply. As to your question, the major benefit for getting USB working would be the following (at least for me):

1. Modern hardware generally supports USB, so you could have a PC or even a Raspberry Pi with LCD screen to send games to your Super UFO.

2. One of the Super UFO's major faults is the fact that it doesn't support long file names. This makes storing any kind of collection on your SD card quite annoying. By making a proper CLI USB transfer tool/program, someone (like myself) could program a front-end or loader that would allow you to pick any game from your library quiet easily. As with point 1, this loader could be ported to any number of platforms that support USB and could even be skinned to look like a regular copier interface too.

Anyway, I understand that not everyone will agree with me or have the same priority or feelings on the mater, but I figure I would put it out there all the same. Smile

EDIT: Here is the documentation and example code. Please add it to ucon! Very Happy

https://github.com/devinacker/roswell/blob/master/docs/usb_upload.txt
https://github.com/devinacker/roswell/blob/master/docs/header.txt

https://github.com/devinacker/roswell/blob/master/scripts/loadrom.py
Back to top
View user's profile Send private message
dbjh



Joined: 02 Aug 2003
Posts: 167

PostPosted: Thu May 17, 2018 4:38 pm    Post subject: Reply with quote

Thank you for your reply and the links!

I had a look at the Python transfer code and it is surprisingly simple. It's the simplest transfer protocol I have seen. Would you be willing to test once I have implemented support?
Back to top
View user's profile Send private message
Trenton_net



Joined: 29 Jun 2006
Posts: 233

PostPosted: Fri May 18, 2018 3:05 pm    Post subject: Reply with quote

@dbjh: The entire USB transfer "protocol" is literally just a bulk transfer of a 64-byte header followed by the entire ROM, so indeed, it is pretty darn simple Smile

In regards to your question, I'd totally be willing to help you get this tested out for sure. All I need to do is dig out my Super Famicom (Not sure which box it's in). Do you normally do your dev/testing on Windows builds first, or Linux?
Back to top
View user's profile Send private message
revenant1



Joined: 25 Mar 2009
Posts: 14

PostPosted: Fri May 18, 2018 4:01 pm    Post subject: Reply with quote

Hi, I'm the author of the aforementioned docs/scripts. I stumbled on this thread after linking them via email a couple of days ago.

I didn't realize the UFO SD's header format had been added to ucon64 since the last time I'd used it - probably would have saved me a little bit of work Smile A couple of bytes are different between my doc and ucon64's, but that could well be an error on my part, since I don't have access to most of my actual SNES carts right now and I may not have tested every possible combination of ROM/RAM settings in the actual firmware...

Most of the bytes in the header for mapping and stuff correspond directly to the UFO's hardware registers, which I wrote a bit more about here, if you're interested.

But yes, as long as you're able to generate a proper header, the USB transfer really is as simple as it looks. I wrote about sending in chunks of "1024 bytes or more" in my notes, since the official tool uses 1024 byte chunks, but any multiple of 64 bytes should work since that's the USB chip's actual buffer size. (Fun fact: the official UFO SD transfer tool is literally just a slightly modified version of one of the example programs provided by the USB chip vendor.)

The main point of that GitHub repo was for me to develop a more advanced way to dump/rewrite actual carts with the UFO SD. It requires running a "server" program of sorts on the SNES, but provides a simple protocol for reading/writing data at any address on the cart as a way of dumping or rewriting things that the actual UFO firmware may not support. (For example, as a test I was able to make a good dump of Kirby Super Star to my computer over USB in about 24 seconds.) The server and its USB protocol are still sort of a work in progress, though.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Trenton_net



Joined: 29 Jun 2006
Posts: 233

PostPosted: Sat May 19, 2018 5:18 am    Post subject: Reply with quote

@revenant1: Ah, your here! Welcome! Perhaps you would be in a better position to help dbjh in doing some testing of uCon64 instead of me. Especially since you know more about what's going on.
Back to top
View user's profile Send private message
revenant1



Joined: 25 Mar 2009
Posts: 14

PostPosted: Sat May 19, 2018 7:39 pm    Post subject: Reply with quote

Sure, I can test it whenever it's available.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Trenton_net



Joined: 29 Jun 2006
Posts: 233

PostPosted: Tue May 22, 2018 7:49 pm    Post subject: Reply with quote

@revenant1: That would be awesome. You just need to check the forums frequently then. Replies are kind of slow, so it's pretty common to post something, then get a reply weeks later and miss it. Very Happy
Back to top
View user's profile Send private message
revenant1



Joined: 25 Mar 2009
Posts: 14

PostPosted: Wed May 23, 2018 4:00 pm    Post subject: Reply with quote

I've got my eye on the thread, but I'm also always reachable via email as well.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
dbjh



Joined: 02 Aug 2003
Posts: 167

PostPosted: Thu May 31, 2018 4:18 pm    Post subject: Reply with quote

I think your data about offset 0x10 and 0x11 is wrong. I guess we can conclude that it is quite unimportant :-) At least the output of ucon64 --dbuh on actual cartridge dumps matches perfectly with the assumption that offset 0x11 represents the internal size in Mbit.

About SRAM sizes. I don't own a Super UFO Pro 8 SD. cage, the guy who started figuring out the UFO SD header structure provided that data. I don't know how he came up with those sizes. I found 128kbit odd and did ask him about it, but didn't get an answer. Are you sure about 3 meaning 256kbit?

About offset 0x13 and 0x14: there are LoROM 8Mbit games that have 0x55 and 0x00 as values. I use an ugly heuristic based on SRAM size and expansion chip which of course works for the data set I have at my disposal. I hope I don't actually have to hard code presets for games.

About offset 0x15: It can also be 0x50 for LoROM 8Mbit games with 8kB SRAM. And it can be 0x20 in LoROM 8Mbit games with only 2kB SRAM (so not more than 32kB SRAM).

I misinterpreted offset 0x17. I will fix that. ucon64 --ufosd produced the right output, but apparently, it has nothing to do with SRAM mapping.

Offset 0x18 was new to me. I just completely overlooked that. Mystic_Merlin on this forum actually provided me with the necessary data (output of ucon64 --dbuh), but I didn't notice it until I read your header document.

About my implementation of --xufosd based on your code: I chose 512 bytes as buffer size because that is what you are using in USBClient.write(). Judging from the speed with which you could dump Kirby Super Star I should probably use a buffer like 32kB. How large can the transfer buffer be?

ufosd_write_rom() uses the first USB configuration. You appear to do the same in USBClient.open(). You are then using Device.get_active_configuration() which appears to involve USB I/O. ufosd_write_rom() just uses the data already available in the struct usb_device. I wonder if what I do is OK.

Do you happen to know whether Windows 8 and newer have built-in support for the WinChipHead CH376 chip? Are drivers available? Any idea about support in Linux? If Linux has support then the code should work there too.
Back to top
View user's profile Send private message
revenant1



Joined: 25 Mar 2009
Posts: 14

PostPosted: Thu May 31, 2018 8:18 pm    Post subject: Reply with quote

dbjh wrote:
I think your data about offset 0x10 and 0x11 is wrong. I guess we can conclude that it is quite unimportant Smile

I don't actually remember if the firmware actually does anything with those bytes or not. I'll have to check again when I'm at home.

dbjh wrote:
About SRAM sizes. I don't own a Super UFO Pro 8 SD. cage, the guy who started figuring out the UFO SD header structure provided that data. I don't know how he came up with those sizes. I found 128kbit odd and did ask him about it, but didn't get an answer. Are you sure about 3 meaning 256kbit?

Fairly sure, yeah. 16k/64k/256k/1M are the only sizes the firmware actually offers as options and the stuff in my doc was obtained through a combination of debugging the actual firmware in an emulator plus playing with the corresponding hardware registers on the real unit. I can double-check this, of course.

dbjh wrote:
About offset 0x13 and 0x14: there are LoROM 8Mbit games that have 0x55 and 0x00 as values. I use an ugly heuristic based on SRAM size and expansion chip which of course works for the data set I have at my disposal. I hope I don't actually have to hard code presets for games.

About offset 0x15: It can also be 0x50 for LoROM 8Mbit games with 8kB SRAM. And it can be 0x20 in LoROM 8Mbit games with only 2kB SRAM (so not more than 32kB SRAM).

Hardcoding presets shouldn't be necessary, but those results seem pretty questionable. 0x13-14 are written to registers that don't appear to affect SRAM mapping at all, and all three of those bytes were documented based on interacting directly with the actual hardware and observing the mapped memory in real time.

For 0x15 it's possible that some LoROM games do map SRAM the way writing 0x20 does. The ">32kb SRAM" reflects how bsnes/higan set it up heuristically based on the SRAM size.

Either way it would be very helpful to know which games you're actually referring to.

dbjh wrote:
About my implementation of --xufosd based on your code: I chose 512 bytes as buffer size because that is what you are using in USBClient.write(). Judging from the speed with which you could dump Kirby Super Star I should probably use a buffer like 32kB. How large can the transfer buffer be?

Any multiple of 64 bytes should be fine, it doesn't actually make that big of a difference. I was only using a small buffer size in the Python client because originally my SNES-side transfer code wasn't very optimized and didn't handle large buffer sizes well. For sending games to the UFO firmware it probably doesn't matter.

dbjh wrote:
ufosd_write_rom() uses the first USB configuration. You appear to do the same in USBClient.open(). You are then using Device.get_active_configuration() which appears to involve USB I/O. ufosd_write_rom() just uses the data already available in the struct usb_device. I wonder if what I do is OK.

The device only has one (bulk) endpoint in either direction so the defaults may be good enough.

Quote:
Do you happen to know whether Windows 8 and newer have built-in support for the WinChipHead CH376 chip? Are drivers available? Any idea about support in Linux? If Linux has support then the code should work there too.

Windows 8 doesn't. Linux probably doesn't either. The drivers are included on the CD that comes with the copier, and from the chip vendor's website, but it's an unsigned driver (so you have to disable that requirement in newer Windows versions before being able to install it) and my own tools currently require using Zadig or some similar tool to add libusb support to the device. There's an actual C header or something available from the vendor that can be used to communicate with the regular driver under Windows but I don't have a link to it at the moment.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
dbjh



Joined: 02 Aug 2003
Posts: 167

PostPosted: Tue Jun 05, 2018 4:16 pm    Post subject: Reply with quote

revenant1 wrote:
dbjh wrote:
I think your data about offset 0x10 and 0x11 is wrong. I guess we can conclude that it is quite unimportant :-)

I don't actually remember if the firmware actually does anything with those bytes or not. I'll have to check again when I'm at home.

The fact that we use completely different values while games run fine is proof that they are at least unimportant. In regs.txt you do not mention that the byte at offset 0x10 is being used.

revenant1 wrote:
dbjh wrote:
About SRAM sizes. I don't own a Super UFO Pro 8 SD. cage, the guy who started figuring out the UFO SD header structure provided that data. I don't know how he came up with those sizes. I found 128kbit odd and did ask him about it, but didn't get an answer. Are you sure about 3 meaning 256kbit?

Fairly sure, yeah. 16k/64k/256k/1M are the only sizes the firmware actually offers as options and the stuff in my doc was obtained through a combination of debugging the actual firmware in an emulator plus playing with the corresponding hardware registers on the real unit. I can double-check this, of course.

I assume you are correct. BTW it is in line with the old Super UFO.

revenant1 wrote:
dbjh wrote:
About offset 0x13 and 0x14: there are LoROM 8Mbit games that have 0x55 and 0x00 as values. I use an ugly heuristic based on SRAM size and expansion chip which of course works for the data set I have at my disposal. I hope I don't actually have to hard code presets for games.

About offset 0x15: It can also be 0x50 for LoROM 8Mbit games with 8kB SRAM. And it can be 0x20 in LoROM 8Mbit games with only 2kB SRAM (so not more than 32kB SRAM).

Hardcoding presets shouldn't be necessary, but those results seem pretty questionable. 0x13-14 are written to registers that don't appear to affect SRAM mapping at all, and all three of those bytes were documented based on interacting directly with the actual hardware and observing the mapped memory in real time.

I understand that 0x13-0x14 don't affect SRAM mapping, that is why I said ugly heuristic. There appeared to be a pattern in the small dataset that I had. I attempted to generalize the pattern. Without knowing what the values actually meant that's about the best one can do.

The LoROM 8Mbit games that have offset 0x13-0x14 set to 0x55 and 0x00 that I know of are "Legend of Zelda, The - A Link to the Past (U) [!]" and "Star Fox (U) (V1.0) [!]". "Legend of Zelda, The - A Link to the Past (U) [!]" also has offset 0x15 set to 0x50.

Here is a list of LoROM games that have less than 32kB SRAM, yet have bit 5 of the byte at offset 0x15 set (0x20/0x60):
Super Tetris 2 + Bombliss - Gentei Han (J) - 8Mbit/2kB SRAM/0x20
Tin Star (U) [!] - 16Mbit/2kB SRAM/0x20
Super Mario All-Stars (U) - 16Mbit/8kB SRAM/0x20
Uniracers - 16Mbit/8kB SRAM/0x20
Super Metroid (JU) [!] - 24Mbit/8kB/0x60

revenant1 wrote:
For 0x15 it's possible that some LoROM games do map SRAM the way writing 0x20 does. The ">32kb SRAM" reflects how bsnes/higan set it up heuristically based on the SRAM size.

Looking at the list above that heuristic appears to be wrong.

revenant1 wrote:
dbjh wrote:
ufosd_write_rom() uses the first USB configuration. You appear to do the same in USBClient.open(). You are then using Device.get_active_configuration() which appears to involve USB I/O. ufosd_write_rom() just uses the data already available in the struct usb_device. I wonder if what I do is OK.

The device only has one (bulk) endpoint in either direction so the defaults may be good enough.

It's not a default, just the first configuration. I don't understand why PyUSB performs USB I/O even when using libusb-0.1, because the configuration data is already present in the struct usb_device.

revenant1 wrote:
dbjh wrote:
Do you happen to know whether Windows 8 and newer have built-in support for the WinChipHead CH376 chip? Are drivers available? Any idea about support in Linux? If Linux has support then the code should work there too.

Windows 8 doesn't. Linux probably doesn't either. The drivers are included on the CD that comes with the copier, and from the chip vendor's website, but it's an unsigned driver (so you have to disable that requirement in newer Windows versions before being able to install it) and my own tools currently require using Zadig or some similar tool to add libusb support to the device.

I didn't know there are that many hoops to jump through. Thanks for mentioning Zadig.

revenant1 wrote:
There's an actual C header or something available from the vendor that can be used to communicate with the regular driver under Windows but I don't have a link to it at the moment.

Have you found that link again?
Back to top
View user's profile Send private message
revenant1



Joined: 25 Mar 2009
Posts: 14

PostPosted: Wed Jun 06, 2018 1:35 am    Post subject: Reply with quote

Quote:
The fact that we use completely different values while games run fine is proof that they are at least unimportant. In regs.txt you do not mention that the byte at offset 0x10 is being used.


Yeah, it's probably not important. I noticed that Super Metroid ends up with the value 0x20 at 0x11, which might just be the ROM size in megabits (0x18) rounded up to the next power of two.

Here are my results (for 0x13-15) for most of the games you listed. These were obtained by loading headerless copies of the ROMs from a SD card, then immediately saving them back with whatever default header settings were applied by the firmware.

Link to the Past: 0x15, 0x28, 0x22
Super Tetris 2 + Bombliss Gentaiban: 0x15, 0x28, 0x11
Tin Star: 0x55, 0x20, 0x11
Super Mario All-Stars: 0x55, 0x20, 0x22
Uniracers: 0x55, 0x20, 0x22
Super Metroid: 0x55, 0x00, 0x62

This mostly all agrees with my docs (note that the upper two bits of 0x15 are based on ROM size, not SRAM size).

The firmware seems to use 0x2x for byte 0x15 when there's at least 8kb of SRAM, rather than over 32kb. This shouldn't matter for most games, but doing it this way may trigger copy protection in some games that try to detect improper SRAM mapping. (Also, the firmware seems to put the same digit in both the high and low nibble of 0x15, but the low nibble doesn't actually do anything, as far as I'm aware. e.g. the values 0x20 and 0x22 should both do the same thing)

Running Star Fox (or any other SuperFX game) on the Super UFO doesn't even work unless you have a real cartridge copy of exactly the same version of the same game attached, because it's not physically possible for the chip to interface with anything other than the actual cartridge that it's on. I wouldn't worry about generating correct headers for these.

dbjh wrote:
Have you found that link again?


Check the bottom of the CH372 page here. CH372EVT.ZIP has the header files and some example code and CH372DRV.EXE is the installers for the drivers and the API DLL.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    ToToTEK.COM Forum Index -> Copiers and Hardware All times are GMT
Goto page 1, 2, 3  Next
Page 1 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group