Using a PIM interface for improving computer medical image processing abilities

3.1PSRAM

PSRAM is a type of DRAM with circuits for self-refresh and address-decode. Therefore, with regard to the memory controller, PSRAM is controlled in the same way as a static random-access memory (SRAM). Unlike most SRAM, however, PSRAM has a WAIT signal. The refresh of typical DRAMs is operated by the memory controller, but since PSRAM has a built-in refresh logic, the response time of PSRAM is changeable. Thus, the WAIT signal informs what is dealing with a required address and command for the memory controller. Figures 2 and 3 illustrate the basic read and write operations of PSRAM.

Figure 2.

Write timing of PSRAM [13].

Figure 3.

Read timing of PSRAM [13].

The control signals of PSRAM are CLK, ADDRESS, ADV#, CE#, OE#, WE#, WAIT, and DQ, shown in Fig. 2. The ADV# indicates that a value of the ADDRESS is valid. The CE is used as a chip select signal. Since the data bus of PSRAM is bi-directional, the OE# expresses the present direction of the data bus. The WE# indicates that a present request is either read or write, and the WE# and the ADV# are asserted at the same time. As the PSRAM protocol is based on variable latency, the WAIT signal indicates the status of the data bus according to the WE#. Because the WE# is the write mode in Fig. 2, WAIT is asserted when the data bus is able to receive the write data. The DQ, the data bus shown in Fig. 2, illustrates the 4-beat burst transfer and write mode. Figure 3 displays the read timing of PSRAM, which is nearly identical to the write timing shown in Fig. 2. As the PSRAM mode in the figure is read, however, WE# is deasserted when the ADV# is asserted. The OE# is contrary to the write mode of PSRAM.

3.2Proposed protocol for PIM

As the processing logic in PIM works according to a read-modify-write scheme (except for basic memory operations), most PIM operations must be atomic [9]. For atomic operations, a packet format of the PIM operation requires various fields such as an opcode, destination and source addresses, and immediate data. Thus, this paper proposes a format of the PIM operation and a method for transferring the PIM operation based on the PSRAM interface. Figure 4 illustrates the format of the proposed PIM-operations.

Figure 4.

Packet format of PIM operations.

The proposed PIM packet consists of an opcode, a destination address, a source address, and immediate data fields. The opcode field indicates an instruction processed by the processing logic. In basic memory operations, the destination address field is used for read and write addresses. The destination address field, however, can also be used for writing the results of PIM operations. If the instruction in the opcode field is an immediate type, the processing logic uses the value of the immediate field in the PIM packet as the operand for processing an instruction; but if the instruction is a recall type, the processing logic uses the operand loaded from the address of the source address field in the PIM packet. To transfer PIM operations through the PSRAM interface, we separate the destination address field from one PIM packet, shown in Fig. 4. Since the address bus and the data bus of the PSRAM interface are divided, we employ the address bus of the PSRAM interface to transfer destination addresses without modification. To transfer the opcode, the source address, and immediate data, we use the transfer mode of the data bus in the PSRAM interface. Under the assumption that the PIM packet has a fixed length, we omit the packet head and tail fields even though they do not have to be omitted. Figure 5 shows the timing of the proposed PIM interface and protocol.

Figure 5.

Proposed PIM interface and protocol.

Figure 6.

Block diagram of the PIM platform.

4.1Experimental environment

To verify the functions and the performance of the PIM interface, we designed an embedded system-based PIM platform that will specifically adapt to medical imaging systems. Figure 6 illustrates a block diagram of the PIM platform. The PIM platform board consists of two parts: a PIM prototype and a base board. The former is a prototype of the 3D-IC-based PIM chip. The FPGA [14] and the SDRAM [15] in the PIM prototype are employed as the processing logic and the DRAM macro in PIM, respectively. A 32-bit commercial application processor (AP) [16] is used as the host processor of the PIM platform. The AP has an external memory channel, referred to as the “external interface module” (EIM). We implement the PIM interface by modifying the PSRAM protocol on the EIM. The bus speed of the EIM in the AP supports a maximum bus speed of 133 MHz, but since we must verify the compatibility with legacy systems and the functions of the proposed PIM interface, we set the EIM bus in the PIM platform at a bus speed of 33.33 MHz.

This study entails the use of 32-MB SDRAM as the internal memory of PIM, and the data bus of the PIM interface in the platform has a 16-bit wide 4-beat burst transfer. Thus, the PIM platform has a destination address of 26 bits and a packet length of 64 bits, a 6-bit opcode, a 26-bit source address, and a 32-bit immediate data packet. Figure 7 shows that the PIM packet is communicated between the PIM prototype and the AP. Because the number of PIM opcodes supported in this paper is limited by hardware resources and the response time, the design of the processing logic allows the processing of simple operations. The opcode list appears in Table 1. The OS in the proposed PIM platform uses a 32-bit embedded Linux.

Figure 7.

Protocol of the proposed PIM interface in the PIM platform.

4.2Experimental results

To verify the proposed PIM interface and operations in the proposed PIM, we developed test applications based on MARCH pattern tests on the MARCH algorithm [17]. Furthermore, recent research [18, 19, 20] related to PIM has reported the suitability of data-intensive applications for PIM such as image processing, artificial intelligence, and graph algorithms. Therefore, we implemented median-filtering processed by the AP and the proposed PIM for data-intensive applications. We processed the MARCH algorithm test-based applications as shown in Fig. 8 and generated the destination and source operands as well as the number of PIM entries. Then each PIM operation in Table 1 was executed by the AP and the proposed PIM. To verify the functions of the proposed PIM interface and PIM, we compared their results. During the experiments, problems of compatibility with legacy systems did not occur, so we confirmed that all functions in the proposed PIM and the interface were normal.

Figure 8.

Flowchart of test operations based on the pattern tests from the MARCH algorithm.

To verify the function and the performance of the proposed PIM, we developed two application programs based on the embedded system with the AP: the median-filter algorithm, executed by only the AP and the same algorithm executed by only the proposed PIM. Each program used its own hardware resources to process one standard-processing image and two medical images, both of which were 24-bit color images with a resolution of 512 by 512 pixels.

To evaluate the performance of the PIM interface, this paper executed the median-filter programs on the embedded system-based PIM platform board and used the functions of window API to measure the processing times of each program [21, 22]. Since the median-filter processing procedures of the AP and the proposed PIM differ in a number of ways, the methods of measurement of the processing times of the programs also differ and are diagrammed in Fig. 9.

Figure 9.

Flowcharts for measuring the performance of the AP and the proposed PIM.

In the case of the AP, the processing time measurement must begin when the test image from the flash memory is loaded into the main memory of the system and end when the resulting image and report files for evaluation are written to the flash memory after median filtering is complete. In the case of PIM, the processing time measurement must begin when the test image from the flash memory is loaded into PIM and end when the resulting image and report files for evaluation are written to PIM after the median filtering is complete. Since a DDR3 is used as the main memory in the PIM platform, and its access time differs from that of the PIM, employing these measuring methods assured the reliability and fairness of the experimental results in this paper.

Figure 10.

Image processing results of the test programs of the AP and the proposed PIM.

The processing results are displayed in Fig. 10. The experiments in this paper used a woman, Lena, cancer, and skin as source images for the median filtering process [23, 24, 25]. Lena is a standard test image in the field of image processing; the cancer image is provided by the Digital Slide Archive, a software platform for cancer research; and the skin image, which shows pigmented skin lesion, comes from HAM10000. In the resulting images processed by the AP and PIM, the edges resulting from median filtering are smoother than those of the source images. In addition, since the resulting images of median filtering by the AP match those by the PIM, we confirm that the operations performed by the proposed PIM interface and PIM functioned normally.

Table 2 compares the median-filter processing times of the AP and the proposed PIM. To create the images of Lena, cancer, and the skin, the processing times of the AP were 8.004326 secs, 6.719369 secs, and 7.862624 secs, respectively, and those of PIM were 1.288066 secs, 1.245627 secs, and 1.287875 secs, respectively. The processing times of PIM were almost the same, but the results of the AP differ, the reason being that the AP and PIM must repeatedly process a sorting algorithm for processing the median filter [26]. To do so, PIM uses a hardware-sorting module, so the comparators in the sorting module have uniform latency regardless of the results of comparison. The sorting algorithm of the AP, however, is processed by software, and the sorting time is affected by the results of the comparison of the features of the test images. Therefore, the sorting time of the AP is not uniform, and the results of the AP in Table 2 vary widely, showing that the processing time of PIM is more than 81% faster than that of the AP. The explanation for this finding is that most of the median filtering of the PIM was processed by hardware-based processing logic and the internal SDRAM. As a result, the internal memory in the PIM reduced the memory-transaction overhead for accessing pixel data, so the processing logic in the PIM accelerated the median filtering process. We repeated the experiments 100 times. Thus, to mitigate interruptions by the background processes of the OS, we averaged the results from the repeated experiments to determine the overall experimental results.